-- Logs begin at Tue 2022-05-31 13:52:06 UTC, end at Tue 2022-05-31 15:31:54 UTC. -- May 31 14:23:47.076966 dv-cinder-839793-14-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 31 14:23:47.084673 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 31 14:23:47.089401 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue May 31 14:23:47 2022] *** May 31 14:23:47.089401 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 31 14:23:47.089401 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 31 14:23:47.089401 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: nodename: dv-cinder-839793-14-iscsi May 31 14:23:47.089401 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: machine: x86_64 May 31 14:23:47.089401 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: clock source: unix May 31 14:23:47.089401 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: pcre jit disabled May 31 14:23:47.089401 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: detected number of CPU cores: 4 May 31 14:23:47.089401 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: current working directory: / May 31 14:23:47.089401 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: detected binary path: /usr/bin/uwsgi-core May 31 14:23:47.089401 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: your processes number limit is 47680 May 31 14:23:47.089401 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: your memory page size is 4096 bytes May 31 14:23:47.089401 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: detected max file descriptor number: 2048 May 31 14:23:47.089401 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: lock engine: pthread robust mutexes May 31 14:23:47.089401 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: thunder lock: enabled May 31 14:23:47.089401 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 31 14:23:47.089401 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 31 14:23:47.134222 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: Python main interpreter initialized at 0x55dad385b9c0 May 31 14:23:47.134326 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: python threads support enabled May 31 14:23:47.134326 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: your server socket listen backlog is limited to 100 connections May 31 14:23:47.134326 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: your mercy for graceful operations on workers is 90 seconds May 31 14:23:47.134562 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: mapped 403077 bytes (393 KB) for 2 cores May 31 14:23:47.134699 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: *** Operational MODE: preforking *** May 31 14:23:47.134995 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: *** uWSGI is running in multiple interpreter mode *** May 31 14:23:47.134995 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: spawned uWSGI master process (pid: 80219) May 31 14:23:47.135200 dv-cinder-839793-14-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 31 14:23:47.135780 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: spawned uWSGI worker 1 (pid: 80233, cores: 1) May 31 14:23:47.136122 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: spawned uWSGI worker 2 (pid: 80234, cores: 1) May 31 14:23:47.136284 dv-cinder-839793-14-iscsi devstack@keystone.service[80219]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 31 14:23:52.850848 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 31 14:23:52.850848 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: from cryptography.utils import int_from_bytes May 31 14:23:52.851626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 31 14:23:52.851626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: from cryptography.utils import int_from_bytes May 31 14:23:52.851852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 31 14:23:52.851852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: from cryptography.utils import int_from_bytes May 31 14:23:52.852302 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 31 14:23:52.852302 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: from cryptography.utils import int_from_bytes May 31 14:23:52.901120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 31 14:23:52.901310 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 31 14:23:52.901501 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] command line args: [] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 31 14:23:52.901728 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 31 14:23:52.901809 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 31 14:23:52.901929 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ================================================================================ {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 31 14:23:52.902014 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 31 14:23:52.902124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] command line args: [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 31 14:23:52.902202 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] admin_token = **** {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.902398 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 31 14:23:52.902484 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] config_dir = [] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.902568 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ================================================================================ {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 31 14:23:52.902674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.902763 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] config_source = [] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.902858 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] admin_token = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.902942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] debug = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.903109 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] config_dir = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.903194 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.903377 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.903498 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.903582 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] config_source = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.903649 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.903717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] debug = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.903851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.903985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.904093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.904197 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.904300 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.904403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] list_limit = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.904498 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] log_config_append = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.904605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.904711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.904819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.904922 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] log_dir = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.905033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.905136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] log_file = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.905231 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_config_append = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.905326 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.905444 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.905570 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.905646 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_dir = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.905762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_file = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.905835 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.905956 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.906039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.906159 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.906240 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.906319 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.906442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.906524 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.906606 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.906726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.906803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.906927 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.907010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.907150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.907266 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.907423 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.907573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.907650 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.907766 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.907850 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.907929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.908071 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.908152 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.908326 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.908410 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.908577 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.908808 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.67/identity {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.908924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.909022 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] publish_errors = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.909152 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.909235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.909436 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.909562 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.67/identity {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.909648 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.909750 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] publish_errors = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.909829 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.909987 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.910153 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.910232 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.910380 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] standard_threads = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.910464 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.910537 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.910734 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.910818 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.910929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.911009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.911123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] use_journal = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.911207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] standard_threads = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.911287 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] use_json = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.911394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.911481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] use_stderr = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.911619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] use_syslog = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.911698 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.911811 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.911894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.911998 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] use_journal = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.912091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.912166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] use_json = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.912310 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.912388 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] use_stderr = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.912510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] use_syslog = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.912598 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.912747 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 31 14:23:52.912826 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.913011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.913089 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.913247 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.913332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.913493 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.913577 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.913728 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.913807 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.913973 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.914123 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.914203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.914274 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.914453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.914531 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.914707 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.914802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.914956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.915036 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.915192 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.915277 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.915437 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.915524 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.915676 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.915769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.915907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.915997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.916154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.916240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.916404 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.916485 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.916650 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.916735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.916886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.916973 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.917136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.917216 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.917393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.917472 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.917642 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.917730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.917882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.917959 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.918110 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.918186 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.918339 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.918425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.918567 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.918683 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.918829 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.918918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.919078 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.919166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.919325 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.919404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.919555 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.919633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.919784 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.919934 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.920032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.920111 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.920277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.920362 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.920510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.920589 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.921261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.921380 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.921478 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.921556 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.921715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.921796 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.921949 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.922027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.922177 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.922256 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.922405 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.922485 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.922634 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.922770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.922882 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.922971 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.923117 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.923273 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.923372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.923533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.923639 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.923735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.923838 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.923937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.924038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.924142 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.924240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.924337 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.924462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.924567 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.924711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.924831 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.924941 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.925019 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.925177 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.925266 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.925411 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.925503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.925647 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.925726 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.925881 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.925961 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.926115 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.926204 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.926346 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.926425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.926594 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.926723 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.926851 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.926930 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.927088 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.927170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.927320 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.927419 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.927574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.927702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.927807 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.927893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.928038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.928129 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.928274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.928352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.928504 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.928585 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.928768 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.928847 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.929005 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.929084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.929247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.929332 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.929488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.929576 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.929724 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.929804 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.929958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.930038 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.930198 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.930277 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.930446 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.930526 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.930701 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.930783 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.930940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.931031 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.931181 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.931260 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.931417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.931495 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.931652 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.931802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.931878 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.931951 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.932110 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.932189 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.932346 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.932433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.932598 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.932677 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.932835 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.932913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.933074 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.933161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.933308 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.933388 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.933549 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.933628 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.933784 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.933871 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.934017 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.934096 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.934251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.934330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.934482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.934569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.934743 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.934840 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.934974 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.935054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.935208 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.935288 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.935436 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.935524 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.935669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.935748 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.935901 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.935979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.936139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.936226 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.936426 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.936994 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.937168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.937248 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.937426 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.937515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.937666 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.937746 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.937897 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] auth.external = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.937978 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.938127 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.938215 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.938415 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.938501 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.938671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.938781 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.938875 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] auth.password = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.938977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.external = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.939112 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] auth.token = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.939203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.939350 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.939431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.939585 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.939665 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.939832 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.939917 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.password = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.940066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.940146 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.token = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.940313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.940394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.940553 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.940633 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.940782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.940868 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.941019 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] credential.caching = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.941100 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.941265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.941347 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.941514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.941593 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.941751 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.941832 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.941985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.942066 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.942228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.942308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.942462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.942550 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.942747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.942831 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.942993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.943072 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.943226 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.943305 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.943470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.943550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.943727 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.943807 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.943979 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.944058 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.944219 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.944299 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.944469 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.944548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.944724 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.944804 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.944960 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] federation.caching = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.945041 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.945196 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.945285 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.945440 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.945526 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.945683 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.945801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.945910 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.945989 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.946154 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.946241 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.946387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.946464 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.946628 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.946750 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.946883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.946963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.947132 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.947213 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.947366 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.947446 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.947610 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.947689 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.947844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.caching = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.947925 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.948090 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.948170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.948331 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.948410 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.948563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.948643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.948796 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.948875 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.949041 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.949120 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.949270 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.949350 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.949505 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.949584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.949744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.949831 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.949989 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.950068 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.950224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.950302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.950450 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.950538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.950698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.950782 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.950943 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.951022 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.951188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.951280 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.951425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.951504 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.951660 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.951740 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.951919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.952045 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.952182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.952262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.952422 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.952508 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.952658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.952747 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.952889 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.953231 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.953389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.953469 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.953620 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.953708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.953854 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.953934 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.954094 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.954174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.954331 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.954419 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.954576 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.954707 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.954830 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.954911 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.955092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.955194 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.955316 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.955396 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.955552 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.955642 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.955792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.955879 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.956035 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.956114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.956264 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.956344 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.956499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.956588 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.956728 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.956815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.956965 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.957046 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.957203 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.957291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.957444 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.957522 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.957677 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.957756 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.957912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.958005 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.958151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.958230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.958393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.958472 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.958624 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.958780 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.958860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.958933 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.959110 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.959190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.959355 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.959443 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.959592 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.959679 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.959827 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.959905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.960065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.960152 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.960324 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.960411 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.960532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.960612 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.960777 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.960895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.961020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.961129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.961255 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.961377 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.961499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.961626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.961872 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.961952 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.962106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.962185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.962336 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.962432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.962576 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.962710 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.962844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.962954 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.963070 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.963183 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.963301 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.963399 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.963547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.963628 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.963789 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.963870 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.964031 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.964110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.964267 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.964345 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.964509 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.964589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.964738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.964818 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.964983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.965063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.965218 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.965300 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.965454 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.965535 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.965690 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.965769 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.965925 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.966006 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.966166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.966246 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.966409 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.966489 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.966656 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.966739 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.966918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.966997 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.967132 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.967231 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.967376 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.967459 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.967651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.967732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.967887 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.967966 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.968125 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.968206 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.968370 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.968449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.968610 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.968688 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.968839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.968919 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.969074 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.969154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.969314 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] resource.caching = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.969395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.969562 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.969643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.969798 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.969877 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.970033 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.970112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.970278 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.970358 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.970525 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.970603 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.970794 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.970875 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.971036 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.971117 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.971270 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.971349 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.971509 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.971589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.971744 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] role.caching = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.971825 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.971991 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] role.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.972070 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.972220 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.972299 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.972466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.972547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.972708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] role.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.972787 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.972949 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.973028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] role.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.973183 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.973262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.973424 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.973503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.973666 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.973746 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.973909 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.973990 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.974153 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.974233 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.974383 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.974470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.974627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.974754 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.974894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.975036 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.975140 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.975233 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.975378 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.975458 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.975634 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.975762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.975889 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.975998 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.976142 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.976245 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.976395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.976494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.976740 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.976828 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.976987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.977068 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.977221 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.977301 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.977473 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.977561 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.977722 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.977803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.977960 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.978040 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.978190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.978278 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.978420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.978501 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.978666 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.978750 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.978921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.979013 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.979171 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.979250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.979410 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.979490 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.979643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.979730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.979879 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.979959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.980121 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.980201 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.980347 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] token.caching = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.980429 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.980583 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.980663 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.980828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.980908 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.981061 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.981141 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.981302 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.981382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.981544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.981624 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.981776 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.981856 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.982016 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.982097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.982257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.982337 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.982500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.982581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.982768 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.982849 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.983006 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.983089 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.983766 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.983851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.984015 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.984096 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.984253 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.984332 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.984478 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.984558 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.984720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.984801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.984968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.985047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.985220 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.985301 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.connection = **** {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.985454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.985543 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.985701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.985782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.985941 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.986020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.986181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.986263 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.connection = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.986417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.986507 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.986698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.986784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.986906 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.986999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.987142 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.987223 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.987384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.987464 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.987618 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.987706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.987860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.987940 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.988093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.988173 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.988339 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.988476 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.988577 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.988658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.988826 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.988906 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.989059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.989213 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.989291 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.989363 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.989524 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.989604 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.989676 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 31 14:23:52.989761 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.989997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.990245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.990488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 31 14:23:52.990678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 31 14:23:53.019105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.019337 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.019562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.020158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.020399 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.020641 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.028011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.029127 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.056500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.056735 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.057582 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.057822 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.111678 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.111924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.160877 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.161113 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.161879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.162121 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.227956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.228184 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.228409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.228513 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.228628 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.228713 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.228845 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.228929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.229059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.229144 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.229292 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.229516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.252535 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.252712 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.252798 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.252949 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.253033 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.253185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.253268 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.253405 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.253488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.253621 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.253705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.253832 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.253924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.254051 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.265233 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.265547 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.277827 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.278060 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.278310 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.278394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.278534 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.278618 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.278788 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.278874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.279005 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.279090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.279218 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.279302 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.279440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.279660 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 31 14:23:53.297398 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.297609 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 31 14:23:53.297693 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.297871 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 31 14:23:53.297954 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.298128 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 31 14:23:53.298210 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.298393 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 31 14:23:53.298474 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.298674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.298759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.298933 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.299015 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.299185 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.299278 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.299444 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.310080 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.310354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.310435 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.310628 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.310739 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.310936 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.311019 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.311286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.319029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.319436 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.322287 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.322565 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.322735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.322869 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.323022 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.323147 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.323309 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.323425 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.323592 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.323795 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.323886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.324066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.324247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.324328 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.324527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.324616 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.324812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.324894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.325087 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.325177 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.325365 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.325638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.352199 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.352717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.353913 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.354188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.354432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.354750 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.360077 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.360362 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.360567 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.360744 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.360872 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.361117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.368183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.368417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.368553 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.368809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.372945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.373296 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.373437 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.373614 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.373751 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.373922 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.374050 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.374200 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.374327 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.374461 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.374588 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.374743 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.374871 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.375011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.375147 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.375332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.393323 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.393598 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.393914 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.394196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.398173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.398951 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.401769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.402062 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.402607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.402931 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.407522 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.408537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.411080 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.412159 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.416097 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.416399 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.416678 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.416954 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.417224 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.417428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.417557 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.417683 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.417835 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.418137 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.418422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.418728 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.436869 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.437146 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.437409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.437676 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.438206 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.438493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.438796 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.439065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.445597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.447029 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.447341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.447617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.448767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.449049 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.450610 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.450921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.451198 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.451473 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.451767 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.452447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.452811 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.453366 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.453661 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.453966 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.465358 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.465632 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.465907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.466183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.466460 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.466769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.467044 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.467365 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.468160 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.468447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.468730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.469011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.469286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.469564 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.486016 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.488335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.489279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.491640 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.492605 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.494187 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.494980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.495869 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.496143 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.496420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.496587 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.498247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.498531 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.498868 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.503825 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.505436 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.506284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.507950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.508624 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.508913 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.509193 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.509465 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.511193 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.511488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.511773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.512050 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.519033 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.519315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.519690 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.521681 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.521972 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.522364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.527765 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.528046 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.528312 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.528582 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.528851 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.529133 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.529403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.529674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.529942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.530793 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.530885 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 31 14:23:53.531063 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.531159 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.531326 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.531410 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.531595 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.531735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.531850 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.531934 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.532215 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.532496 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.532773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.533046 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.533318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.533592 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.533870 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.534144 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.534405 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 31 14:23:53.569276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55dad385b9c0 pid: 80234 (default app) May 31 14:23:53.572114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55dad385b9c0 pid: 80233 (default app) May 31 14:23:53.572532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e22923-4dca-4c10-9225-a14d2062be71 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:23:53.572823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e22923-4dca-4c10-9225-a14d2062be71 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:23:53.573098 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e22923-4dca-4c10-9225-a14d2062be71 None None] PATH_INFO: `/v3` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:23:53.577879 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1/1] 10.222.0.67 () {50 vars in 791 bytes} [Tue May 31 14:23:53 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:01.601136 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7386f6-5ed3-41cc-8c96-880fbfe041ff None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:01.601661 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7386f6-5ed3-41cc-8c96-880fbfe041ff None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:01.601765 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7386f6-5ed3-41cc-8c96-880fbfe041ff None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:01.606070 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1/2] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:01.611393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed03df-a4ba-497a-877f-e431ae6a6cd6 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:01.611814 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed03df-a4ba-497a-877f-e431ae6a6cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:01.611940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed03df-a4ba-497a-877f-e431ae6a6cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:01.679596 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: INFO dbcounter [None req-caed03df-a4ba-497a-877f-e431ae6a6cd6 None None] Registered counter for database keystone May 31 14:24:01.700694 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG oslo_db.sqlalchemy.engines [None req-caed03df-a4ba-497a-877f-e431ae6a6cd6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80234) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 31 14:24:01.707695 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writer thread running {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 31 14:24:01.992314 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG passlib.handlers.bcrypt [None req-caed03df-a4ba-497a-877f-e431ae6a6cd6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80234) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 31 14:24:01.992645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG passlib.handlers.bcrypt [None req-caed03df-a4ba-497a-877f-e431ae6a6cd6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80234) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 31 14:24:02.048054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-caed03df-a4ba-497a-877f-e431ae6a6cd6 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:02.049659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-caed03df-a4ba-497a-877f-e431ae6a6cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:02.174508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 2/3] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 564 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 31 14:24:02.330798 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7386f6-5ed3-41cc-8c96-880fbfe041ff None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:02.399885 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: INFO dbcounter [None req-bb7386f6-5ed3-41cc-8c96-880fbfe041ff None None] Registered counter for database keystone May 31 14:24:02.418876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG oslo_db.sqlalchemy.engines [None req-bb7386f6-5ed3-41cc-8c96-880fbfe041ff None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80233) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 31 14:24:02.424976 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writer thread running {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 31 14:24:02.647338 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7386f6-5ed3-41cc-8c96-880fbfe041ff None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:02.651926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ff88f-ccf2-4b13-b9a6-bc37cce925c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:02.652597 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ff88f-ccf2-4b13-b9a6-bc37cce925c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:02.652886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ff88f-ccf2-4b13-b9a6-bc37cce925c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:02.653169 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ff88f-ccf2-4b13-b9a6-bc37cce925c0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:02.661389 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ff88f-ccf2-4b13-b9a6-bc37cce925c0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:02.716735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: WARNING keystone.common.rbac_enforcer.enforcer [None req-da6ff88f-ccf2-4b13-b9a6-bc37cce925c0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 31 14:24:02.717083 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ff88f-ccf2-4b13-b9a6-bc37cce925c0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:02.727207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 2/4] 10.222.0.67 () {58 vars in 1324 bytes} [Tue May 31 14:24:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 31 14:24:02.733801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed03df-a4ba-497a-877f-e431ae6a6cd6 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:02.748460 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed03df-a4ba-497a-877f-e431ae6a6cd6 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:02.753467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab854a36-3486-4ea5-804a-ccd538ff4e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:02.754596 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab854a36-3486-4ea5-804a-ccd538ff4e86 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:02.754918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab854a36-3486-4ea5-804a-ccd538ff4e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:02.755197 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab854a36-3486-4ea5-804a-ccd538ff4e86 admin admin] PATH_INFO: `/v3/projects/929ceca16868457ca23cb37daf0ee36d` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:02.756700 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab854a36-3486-4ea5-804a-ccd538ff4e86 admin admin] RBAC: Authorizing `identity:get_project(project_id=929ceca16868457ca23cb37daf0ee36d)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:02.808485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ab854a36-3486-4ea5-804a-ccd538ff4e86 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 31 14:24:02.808834 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab854a36-3486-4ea5-804a-ccd538ff4e86 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:02.814153 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 3/5] 10.222.0.67 () {56 vars in 1207 bytes} [Tue May 31 14:24:02 2022] GET /identity/v3/projects/929ceca16868457ca23cb37daf0ee36d => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:05.013298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56152e5c-e4a1-4a48-9ba8-47b5541a40dc None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:05.013833 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56152e5c-e4a1-4a48-9ba8-47b5541a40dc None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:05.013833 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56152e5c-e4a1-4a48-9ba8-47b5541a40dc None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:05.018216 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 3/6] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 31 14:24:05.023764 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0ca05-4637-4878-89c3-e29999c01769 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:05.024178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0ca05-4637-4878-89c3-e29999c01769 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:05.024326 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0ca05-4637-4878-89c3-e29999c01769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:05.042906 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-c7b0ca05-4637-4878-89c3-e29999c01769 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:05.044428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c7b0ca05-4637-4878-89c3-e29999c01769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:05.052795 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 4/7] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 31 14:24:05.074994 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56152e5c-e4a1-4a48-9ba8-47b5541a40dc None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:05.087310 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56152e5c-e4a1-4a48-9ba8-47b5541a40dc None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:05.091446 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058b425-2e32-4dd6-9675-b7092ded1976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:05.092073 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058b425-2e32-4dd6-9675-b7092ded1976 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:05.092850 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058b425-2e32-4dd6-9675-b7092ded1976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:05.093127 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058b425-2e32-4dd6-9675-b7092ded1976 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:05.100647 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d058b425-2e32-4dd6-9675-b7092ded1976 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:05.102781 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d058b425-2e32-4dd6-9675-b7092ded1976 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:05.111933 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 4/8] 10.222.0.67 () {58 vars in 1324 bytes} [Tue May 31 14:24:05 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 31 14:24:05.117144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0ca05-4637-4878-89c3-e29999c01769 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:05.126264 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0ca05-4637-4878-89c3-e29999c01769 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:05.130760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec2b6a-42f7-47f8-bd87-5b7b92ce6e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:05.131626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec2b6a-42f7-47f8-bd87-5b7b92ce6e93 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:05.131914 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec2b6a-42f7-47f8-bd87-5b7b92ce6e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:05.132196 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec2b6a-42f7-47f8-bd87-5b7b92ce6e93 admin admin] PATH_INFO: `/v3/users/c01eab2db7454d578ce76aa7d022d460` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:05.133603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec2b6a-42f7-47f8-bd87-5b7b92ce6e93 admin admin] RBAC: Authorizing `identity:get_user(user_id=c01eab2db7454d578ce76aa7d022d460)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:05.135477 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec2b6a-42f7-47f8-bd87-5b7b92ce6e93 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:05.137068 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 5/9] 10.222.0.67 () {56 vars in 1198 bytes} [Tue May 31 14:24:05 2022] GET /identity/v3/users/c01eab2db7454d578ce76aa7d022d460 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:07.421971 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d0fe6-9cb0-4c89-89f1-7f65f094e31c None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:07.422462 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d0fe6-9cb0-4c89-89f1-7f65f094e31c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:07.422462 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d0fe6-9cb0-4c89-89f1-7f65f094e31c None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:07.423846 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 5/10] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 31 14:24:07.429083 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c556593-0c8a-4a53-ae09-21dfc934365c None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:07.429482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c556593-0c8a-4a53-ae09-21dfc934365c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:07.429724 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c556593-0c8a-4a53-ae09-21dfc934365c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:07.446657 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-1c556593-0c8a-4a53-ae09-21dfc934365c None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:07.447861 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1c556593-0c8a-4a53-ae09-21dfc934365c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:07.477043 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a66789-2122-4ae5-bf99-f7afd8919455 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:07.477339 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a66789-2122-4ae5-bf99-f7afd8919455 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:07.477629 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a66789-2122-4ae5-bf99-f7afd8919455 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:07.479022 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 6/11] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:07.483820 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977d438-d1d5-4f76-b8be-2d19ff38a08d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:07.484118 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977d438-d1d5-4f76-b8be-2d19ff38a08d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:07.484405 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977d438-d1d5-4f76-b8be-2d19ff38a08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:07.525248 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG passlib.handlers.bcrypt [None req-8977d438-d1d5-4f76-b8be-2d19ff38a08d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80233) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 31 14:24:07.525674 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG passlib.handlers.bcrypt [None req-8977d438-d1d5-4f76-b8be-2d19ff38a08d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80233) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 31 14:24:07.528197 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 6/12] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:07.551048 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c556593-0c8a-4a53-ae09-21dfc934365c None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:07.572323 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8977d438-d1d5-4f76-b8be-2d19ff38a08d None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:07.573908 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8977d438-d1d5-4f76-b8be-2d19ff38a08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:07.574305 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:07.656148 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c556593-0c8a-4a53-ae09-21dfc934365c None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:07.675945 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 7/13] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 31 14:24:07.691741 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f714bd-1499-43cb-b742-a3cf7ad658f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:07.692546 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f714bd-1499-43cb-b742-a3cf7ad658f4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:07.692831 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f714bd-1499-43cb-b742-a3cf7ad658f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:07.693110 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f714bd-1499-43cb-b742-a3cf7ad658f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:07.698262 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977d438-d1d5-4f76-b8be-2d19ff38a08d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:07.705512 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f714bd-1499-43cb-b742-a3cf7ad658f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:07.716562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-34f714bd-1499-43cb-b742-a3cf7ad658f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:24:07.716562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:07.716562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:07.716562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:07.716562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:07.716562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:07.716562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:07.716562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:07.716562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:07.716562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:07.716562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:07.716562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:07.716562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:07.716562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:07.716562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:07.716562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:07.717587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:07.717587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:07.717587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:07.717587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:07.717587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:07.717587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:07.717587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:07.717587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:07.717587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:07.717587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:07.717587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:07.717587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:24:07.717587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:07.717587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:07.717587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:07.717587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:07.718409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:07.718409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:07.718409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:07.718409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:07.718409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:07.718409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:07.718409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:07.718409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:07.718409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:07.718409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:07.718409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:24:07.718409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:24:07.718409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 7/14] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:24:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:24:07.723635 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f714bd-1499-43cb-b742-a3cf7ad658f4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:07.798437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977d438-d1d5-4f76-b8be-2d19ff38a08d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:07.800717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:07.802875 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:07.821665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f714bd-1499-43cb-b742-a3cf7ad658f4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:07.835672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cdf6a-5c52-47f7-98eb-7a98e7831940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:07.836574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cdf6a-5c52-47f7-98eb-7a98e7831940 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:07.836865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cdf6a-5c52-47f7-98eb-7a98e7831940 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:07.837147 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cdf6a-5c52-47f7-98eb-7a98e7831940 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:07.854502 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86d5ee-bcac-4f1c-a1bd-e2ca4ec553d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:07.855182 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86d5ee-bcac-4f1c-a1bd-e2ca4ec553d2 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:07.855465 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86d5ee-bcac-4f1c-a1bd-e2ca4ec553d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:07.855737 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86d5ee-bcac-4f1c-a1bd-e2ca4ec553d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:07.856477 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86d5ee-bcac-4f1c-a1bd-e2ca4ec553d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:07.856789 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86d5ee-bcac-4f1c-a1bd-e2ca4ec553d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:07.858723 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86d5ee-bcac-4f1c-a1bd-e2ca4ec553d2 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:07.871943 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 8/15] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:24:07.877968 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86d5ee-bcac-4f1c-a1bd-e2ca4ec553d2 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:07.906484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0cdf6a-5c52-47f7-98eb-7a98e7831940 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:07.908047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0cdf6a-5c52-47f7-98eb-7a98e7831940 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:07.976719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86d5ee-bcac-4f1c-a1bd-e2ca4ec553d2 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:08.008000 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 8/16] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:24:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 31 14:24:08.010604 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5014cf02-a09d-44e3-9ff5-d2bab519796c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:08.011582 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5014cf02-a09d-44e3-9ff5-d2bab519796c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:08.011870 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5014cf02-a09d-44e3-9ff5-d2bab519796c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:08.012153 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5014cf02-a09d-44e3-9ff5-d2bab519796c None admin] PATH_INFO: `/v3/users/c01eab2db7454d578ce76aa7d022d460` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:08.013379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5014cf02-a09d-44e3-9ff5-d2bab519796c None admin] RBAC: Authorizing `identity:get_user(user_id=c01eab2db7454d578ce76aa7d022d460)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:08.013534 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cdf6a-5c52-47f7-98eb-7a98e7831940 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:08.014899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5014cf02-a09d-44e3-9ff5-d2bab519796c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:08.016188 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:08.016526 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 9/17] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:07 2022] GET /identity/v3/users/c01eab2db7454d578ce76aa7d022d460 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:24:08.022132 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5014cf02-a09d-44e3-9ff5-d2bab519796c None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:08.036263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:08.111054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cdf6a-5c52-47f7-98eb-7a98e7831940 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:08.121363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5014cf02-a09d-44e3-9ff5-d2bab519796c None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:08.148371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd734b-c8ae-45d8-80d7-730c817d0192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:08.149173 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd734b-c8ae-45d8-80d7-730c817d0192 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:08.149470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd734b-c8ae-45d8-80d7-730c817d0192 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:08.149752 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd734b-c8ae-45d8-80d7-730c817d0192 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:08.155956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96e0e2-8f68-4272-b81f-b03f3e2ba24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:08.156752 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96e0e2-8f68-4272-b81f-b03f3e2ba24f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:08.157037 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96e0e2-8f68-4272-b81f-b03f3e2ba24f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:08.157310 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96e0e2-8f68-4272-b81f-b03f3e2ba24f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:08.158476 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c96e0e2-8f68-4272-b81f-b03f3e2ba24f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:08.159959 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c96e0e2-8f68-4272-b81f-b03f3e2ba24f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:08.161546 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 10/18] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:24:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:24:08.167120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96e0e2-8f68-4272-b81f-b03f3e2ba24f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:08.168327 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fd734b-c8ae-45d8-80d7-730c817d0192 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:08.169801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fd734b-c8ae-45d8-80d7-730c817d0192 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:08.191341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-c7fd734b-c8ae-45d8-80d7-730c817d0192 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:24:08.191341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:08.191341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:08.191341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:08.191341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:08.191341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:08.191341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:08.191341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:08.191341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:08.191341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:08.191341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:08.191341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:08.191341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:24:08.191341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:24:08.191341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:24:08.192198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:24:08.192198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:08.192198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:08.192198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:08.192198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:08.192198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:08.192198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:08.192198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:08.192198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:24:08.192198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:08.192198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:08.192198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:08.192198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:08.192198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:08.192198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:08.192198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:08.192198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:08.193009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:24:08.193009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:24:08.193009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:24:08.193009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:24:08.193009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 9/19] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:24:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:24:08.198395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd734b-c8ae-45d8-80d7-730c817d0192 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:08.246370 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:08.256597 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:08.267786 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96e0e2-8f68-4272-b81f-b03f3e2ba24f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:08.299907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd734b-c8ae-45d8-80d7-730c817d0192 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:08.301038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413c717-1338-43f5-8154-ce2fa46fa39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:08.301693 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413c717-1338-43f5-8154-ce2fa46fa39c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:08.301981 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413c717-1338-43f5-8154-ce2fa46fa39c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:08.302259 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413c717-1338-43f5-8154-ce2fa46fa39c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:08.303127 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413c717-1338-43f5-8154-ce2fa46fa39c None admin] RBAC: Adding query filter params (role.id=e8d679596ffb45939229d1e2a13ce9b0, scope.domain.id=default, user.id=c01eab2db7454d578ce76aa7d022d460) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:08.303440 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413c717-1338-43f5-8154-ce2fa46fa39c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:08.304881 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413c717-1338-43f5-8154-ce2fa46fa39c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:08.312032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 11/20] 10.222.0.67 () {56 vars in 1322 bytes} [Tue May 31 14:24:08 2022] GET /identity/v3/role_assignments?user.id=c01eab2db7454d578ce76aa7d022d460&scope.domain.id=default&role.id=e8d679596ffb45939229d1e2a13ce9b0 => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:24:08.335802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d4c51-0ec9-4cda-a986-1d61cd4e7806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:08.336498 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d4c51-0ec9-4cda-a986-1d61cd4e7806 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:08.336789 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d4c51-0ec9-4cda-a986-1d61cd4e7806 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:08.337074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d4c51-0ec9-4cda-a986-1d61cd4e7806 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:08.337824 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d4c51-0ec9-4cda-a986-1d61cd4e7806 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:08.338140 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d4c51-0ec9-4cda-a986-1d61cd4e7806 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:08.339628 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d4c51-0ec9-4cda-a986-1d61cd4e7806 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:08.359230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 10/21] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:24:11.332701 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490dc79-9271-4228-855d-ea60c024cc56 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:11.333556 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490dc79-9271-4228-855d-ea60c024cc56 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:11.333970 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490dc79-9271-4228-855d-ea60c024cc56 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:11.335514 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 12/22] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:11.340987 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81ceb5-f346-43d3-b4a7-9a82af44a03c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:11.341746 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81ceb5-f346-43d3-b4a7-9a82af44a03c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:11.342204 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81ceb5-f346-43d3-b4a7-9a82af44a03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:11.367609 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ca81ceb5-f346-43d3-b4a7-9a82af44a03c None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:11.369037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ca81ceb5-f346-43d3-b4a7-9a82af44a03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:11.436232 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d260d-85ee-47f1-8e2f-b82c4aadb89d None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:11.436737 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d260d-85ee-47f1-8e2f-b82c4aadb89d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:11.439403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d260d-85ee-47f1-8e2f-b82c4aadb89d None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:11.442157 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 13/23] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:11.446737 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4c0c0-b4f9-41fa-94a3-411f1b0634e3 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:11.447201 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4c0c0-b4f9-41fa-94a3-411f1b0634e3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:11.447619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4c0c0-b4f9-41fa-94a3-411f1b0634e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:11.475187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 11/24] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:11.476763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425abc39-50b0-4ced-acab-31bf0f0aeea6 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:11.477776 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425abc39-50b0-4ced-acab-31bf0f0aeea6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:11.478858 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-56e4c0c0-b4f9-41fa-94a3-411f1b0634e3 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:11.479296 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425abc39-50b0-4ced-acab-31bf0f0aeea6 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:11.480631 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-56e4c0c0-b4f9-41fa-94a3-411f1b0634e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:11.482036 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 12/25] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:11.489289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddf2ff-ca22-4ef0-8acd-11245a61c742 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:11.489805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddf2ff-ca22-4ef0-8acd-11245a61c742 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:11.490223 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddf2ff-ca22-4ef0-8acd-11245a61c742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:11.519543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b1ddf2ff-ca22-4ef0-8acd-11245a61c742 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:11.520758 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b1ddf2ff-ca22-4ef0-8acd-11245a61c742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:11.606120 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:11.618012 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 14/26] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 31 14:24:11.619109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4c0c0-b4f9-41fa-94a3-411f1b0634e3 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:11.649525 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 13/27] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:11.650941 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdcd7f-fb14-4a11-9f18-44fd475a7a8b None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:11.651235 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdcd7f-fb14-4a11-9f18-44fd475a7a8b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:11.651513 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdcd7f-fb14-4a11-9f18-44fd475a7a8b None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:11.653060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 14/28] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:11.653757 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bdcd7f-fb14-4a11-9f18-44fd475a7a8b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:11.701660 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:11.729239 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4c0c0-b4f9-41fa-94a3-411f1b0634e3 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:11.762945 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bdcd7f-fb14-4a11-9f18-44fd475a7a8b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:11.766260 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69946d-c3e5-4f52-9d95-c386c06eb1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:11.767288 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69946d-c3e5-4f52-9d95-c386c06eb1ec None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:11.767746 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69946d-c3e5-4f52-9d95-c386c06eb1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:11.768168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69946d-c3e5-4f52-9d95-c386c06eb1ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:11.779532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b69946d-c3e5-4f52-9d95-c386c06eb1ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:11.781198 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b69946d-c3e5-4f52-9d95-c386c06eb1ec None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:11.799874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e76257-2252-4c68-849d-39998ba4dcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:11.800552 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e76257-2252-4c68-849d-39998ba4dcb8 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:11.800745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-5b69946d-c3e5-4f52-9d95-c386c06eb1ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:24:11.800745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:11.800745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:11.800745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:11.800745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:11.800745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:11.800745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:11.800745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:11.800745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:11.800745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:11.800745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:11.800745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:11.800745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:24:11.800745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:24:11.800745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:24:11.801672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:24:11.801672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:11.801672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:11.801672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:11.801672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:11.801672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:11.801672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:11.801672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:11.801672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:24:11.801672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:11.801672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:11.801672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:11.801672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:11.801672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:11.801672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:11.801672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:11.801672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:11.802547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:24:11.802547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:24:11.802547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:24:11.802547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:24:11.802774 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e76257-2252-4c68-849d-39998ba4dcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:11.802774 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e76257-2252-4c68-849d-39998ba4dcb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:11.802774 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e76257-2252-4c68-849d-39998ba4dcb8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:11.803245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e76257-2252-4c68-849d-39998ba4dcb8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:11.804392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 15/29] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:24:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:11.805738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c085069-eca6-4e79-bb20-d8b4a17dc923 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:11.806165 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c085069-eca6-4e79-bb20-d8b4a17dc923 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:11.806586 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c085069-eca6-4e79-bb20-d8b4a17dc923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:11.826947 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6c085069-eca6-4e79-bb20-d8b4a17dc923 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:11.828604 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6c085069-eca6-4e79-bb20-d8b4a17dc923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:11.898288 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 16/30] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:11.899481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c085069-eca6-4e79-bb20-d8b4a17dc923 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:11.942998 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 15/31] 10.222.0.67 () {60 vars in 1131 bytes} [Tue May 31 14:24:11 2022] POST /identity/v3/roles => generated 219 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:24:11.944284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e76257-2252-4c68-849d-39998ba4dcb8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:11.958909 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:11.989979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:12.009011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c085069-eca6-4e79-bb20-d8b4a17dc923 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:12.050676 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d8130-241f-47e3-a6c2-dd2eb91de857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:12.051683 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d8130-241f-47e3-a6c2-dd2eb91de857 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:12.051877 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d8130-241f-47e3-a6c2-dd2eb91de857 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:12.052148 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d8130-241f-47e3-a6c2-dd2eb91de857 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:12.063771 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d8130-241f-47e3-a6c2-dd2eb91de857 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:12.072091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-9a5d8130-241f-47e3-a6c2-dd2eb91de857 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:24:12.072091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:12.072091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:12.072091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:12.072091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:12.072091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:12.072091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:12.072091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:12.072091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:12.072091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:12.072091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:12.072091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:12.072091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:12.072091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:12.072091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:12.072091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:12.073029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:12.073029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:12.073029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:12.073029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:12.073029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:12.073029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:12.073029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:12.073029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:12.073029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:12.073029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:12.073029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:12.073029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:24:12.073029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:12.073029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:12.073029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:12.073029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:12.073782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:12.073782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:12.073782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:12.073782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:12.073782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:12.073782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:12.073782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:12.073782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:12.073782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:12.073782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:12.073782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:24:12.073782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:24:12.073782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 17/32] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:24:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:12.074407 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d8130-241f-47e3-a6c2-dd2eb91de857 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:12.075286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e76257-2252-4c68-849d-39998ba4dcb8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:12.119674 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bba83b-899d-4269-919d-c5566d77c405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:12.120782 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bba83b-899d-4269-919d-c5566d77c405 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:12.121369 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bba83b-899d-4269-919d-c5566d77c405 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:12.121857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bba83b-899d-4269-919d-c5566d77c405 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:12.122821 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bba83b-899d-4269-919d-c5566d77c405 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:12.123509 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bba83b-899d-4269-919d-c5566d77c405 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:12.125351 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bba83b-899d-4269-919d-c5566d77c405 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:12.138938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 16/33] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:12.140161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bba83b-899d-4269-919d-c5566d77c405 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:12.185843 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d8130-241f-47e3-a6c2-dd2eb91de857 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:12.203072 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:12.252631 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39998705-5e32-4e9d-8b95-fe44e97e6808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:12.253243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39998705-5e32-4e9d-8b95-fe44e97e6808 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:12.253532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39998705-5e32-4e9d-8b95-fe44e97e6808 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:12.253800 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:12.253902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39998705-5e32-4e9d-8b95-fe44e97e6808 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:12.255026 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39998705-5e32-4e9d-8b95-fe44e97e6808 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:12.256538 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39998705-5e32-4e9d-8b95-fe44e97e6808 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:12.260974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bba83b-899d-4269-919d-c5566d77c405 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:12.303659 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8ad96-3651-4095-903a-64ad1e06244e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:12.304370 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8ad96-3651-4095-903a-64ad1e06244e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:12.304658 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8ad96-3651-4095-903a-64ad1e06244e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:12.304936 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8ad96-3651-4095-903a-64ad1e06244e None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:12.305698 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e8ad96-3651-4095-903a-64ad1e06244e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:12.306010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e8ad96-3651-4095-903a-64ad1e06244e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:12.307448 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e8ad96-3651-4095-903a-64ad1e06244e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:12.319975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 17/34] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:12.321615 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8ad96-3651-4095-903a-64ad1e06244e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:12.439699 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8ad96-3651-4095-903a-64ad1e06244e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:12.442634 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 18/35] 10.222.0.67 () {60 vars in 1131 bytes} [Tue May 31 14:24:12 2022] POST /identity/v3/roles => generated 225 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:24:12.444127 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39998705-5e32-4e9d-8b95-fe44e97e6808 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:12.504913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da5768-fc3d-4c47-947e-3aa62158d7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:12.505823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da5768-fc3d-4c47-947e-3aa62158d7e4 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:12.506370 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da5768-fc3d-4c47-947e-3aa62158d7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:12.506821 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da5768-fc3d-4c47-947e-3aa62158d7e4 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:12.508104 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da5768-fc3d-4c47-947e-3aa62158d7e4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:12.509784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da5768-fc3d-4c47-947e-3aa62158d7e4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:12.551241 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 18/36] 10.222.0.67 () {60 vars in 1141 bytes} [Tue May 31 14:24:12 2022] POST /identity/v3/projects => generated 301 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:24:12.553626 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b6f9f-e1fc-4653-a2c3-4f91387db32c None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:12.554122 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b6f9f-e1fc-4653-a2c3-4f91387db32c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:12.554894 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b6f9f-e1fc-4653-a2c3-4f91387db32c None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:12.558634 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 19/37] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:12.565143 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a348db-1048-426b-90f4-b4f52bc4e594 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:12.565720 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a348db-1048-426b-90f4-b4f52bc4e594 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:12.566164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a348db-1048-426b-90f4-b4f52bc4e594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:12.593082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-50a348db-1048-426b-90f4-b4f52bc4e594 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:12.595891 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-50a348db-1048-426b-90f4-b4f52bc4e594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:12.598346 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39998705-5e32-4e9d-8b95-fe44e97e6808 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:12.600315 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:12.644297 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb4e51-efb5-49dc-8fb2-92a142ba0de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:12.645455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb4e51-efb5-49dc-8fb2-92a142ba0de3 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:12.645887 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb4e51-efb5-49dc-8fb2-92a142ba0de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:12.646309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb4e51-efb5-49dc-8fb2-92a142ba0de3 None admin] PATH_INFO: `/v3/users/c01eab2db7454d578ce76aa7d022d460` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:12.648007 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb4e51-efb5-49dc-8fb2-92a142ba0de3 None admin] RBAC: Authorizing `identity:get_user(user_id=c01eab2db7454d578ce76aa7d022d460)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:12.649812 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb4e51-efb5-49dc-8fb2-92a142ba0de3 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:12.651753 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 19/38] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:12 2022] GET /identity/v3/users/c01eab2db7454d578ce76aa7d022d460 => generated 247 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:12.659827 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb4e51-efb5-49dc-8fb2-92a142ba0de3 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:12.688779 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 20/39] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:12.727195 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a348db-1048-426b-90f4-b4f52bc4e594 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:12.753220 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:12.828472 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb4e51-efb5-49dc-8fb2-92a142ba0de3 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:12.870918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a348db-1048-426b-90f4-b4f52bc4e594 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:12.872327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768b4e0-b7fe-4945-a2e0-ebb749ebe7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:12.873122 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768b4e0-b7fe-4945-a2e0-ebb749ebe7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:12.873434 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768b4e0-b7fe-4945-a2e0-ebb749ebe7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:12.873741 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768b4e0-b7fe-4945-a2e0-ebb749ebe7b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:12.875043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768b4e0-b7fe-4945-a2e0-ebb749ebe7b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:12.876503 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768b4e0-b7fe-4945-a2e0-ebb749ebe7b1 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:12.878129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 20/40] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:24:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:12.879622 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0627c56-db28-4ede-a1c1-0bab0757ea13 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:12.879915 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0627c56-db28-4ede-a1c1-0bab0757ea13 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:12.880387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0627c56-db28-4ede-a1c1-0bab0757ea13 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:12.882032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 21/41] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:12.885412 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0627c56-db28-4ede-a1c1-0bab0757ea13 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:12.913888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80582af8-c24b-411b-9bc5-17e17987f48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:12.914572 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80582af8-c24b-411b-9bc5-17e17987f48c None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:12.914958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80582af8-c24b-411b-9bc5-17e17987f48c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:12.915239 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80582af8-c24b-411b-9bc5-17e17987f48c None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:12.916031 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80582af8-c24b-411b-9bc5-17e17987f48c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:12.917416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80582af8-c24b-411b-9bc5-17e17987f48c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:12.926529 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:12.939879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 21/42] 10.222.0.67 () {60 vars in 1131 bytes} [Tue May 31 14:24:12 2022] POST /identity/v3/roles => generated 223 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:24:12.939879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced04b00-bc17-443e-80d1-06b0fae4f300 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:12.939879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced04b00-bc17-443e-80d1-06b0fae4f300 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:12.939879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced04b00-bc17-443e-80d1-06b0fae4f300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:12.953014 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ced04b00-bc17-443e-80d1-06b0fae4f300 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:12.954265 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ced04b00-bc17-443e-80d1-06b0fae4f300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:13.012446 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0627c56-db28-4ede-a1c1-0bab0757ea13 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:13.027828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:13.032327 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 22/43] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:13.053200 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91201f59-03a5-4833-9cbb-a8ae63b970f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:13.054112 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91201f59-03a5-4833-9cbb-a8ae63b970f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:13.054401 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91201f59-03a5-4833-9cbb-a8ae63b970f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:13.054712 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91201f59-03a5-4833-9cbb-a8ae63b970f2 None admin] PATH_INFO: `/v3/domains/default/users/c01eab2db7454d578ce76aa7d022d460/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:13.055384 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced04b00-bc17-443e-80d1-06b0fae4f300 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:13.056852 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91201f59-03a5-4833-9cbb-a8ae63b970f2 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c01eab2db7454d578ce76aa7d022d460, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:13.058354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91201f59-03a5-4833-9cbb-a8ae63b970f2 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:13.076166 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 22/44] 10.222.0.67 () {58 vars in 1361 bytes} [Tue May 31 14:24:12 2022] PUT /identity/v3/domains/default/users/c01eab2db7454d578ce76aa7d022d460/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:24:13.181798 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced04b00-bc17-443e-80d1-06b0fae4f300 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:13.214509 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:13.221633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731fc55-55a8-40e2-ba68-d21f1d9855ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:13.222464 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731fc55-55a8-40e2-ba68-d21f1d9855ff None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:13.222787 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731fc55-55a8-40e2-ba68-d21f1d9855ff None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:13.223071 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731fc55-55a8-40e2-ba68-d21f1d9855ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:13.224333 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731fc55-55a8-40e2-ba68-d21f1d9855ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:13.225823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731fc55-55a8-40e2-ba68-d21f1d9855ff None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:13.227550 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 23/45] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:24:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:13.233351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91201f59-03a5-4833-9cbb-a8ae63b970f2 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:13.336197 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91201f59-03a5-4833-9cbb-a8ae63b970f2 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:13.368789 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff92aa-be6d-4d7e-85a0-0f6017bc4033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:13.369469 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff92aa-be6d-4d7e-85a0-0f6017bc4033 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:13.369755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff92aa-be6d-4d7e-85a0-0f6017bc4033 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:13.370038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff92aa-be6d-4d7e-85a0-0f6017bc4033 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:13.370806 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff92aa-be6d-4d7e-85a0-0f6017bc4033 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:13.372278 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff92aa-be6d-4d7e-85a0-0f6017bc4033 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:13.401051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 23/46] 10.222.0.67 () {60 vars in 1141 bytes} [Tue May 31 14:24:13 2022] POST /identity/v3/projects => generated 312 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:24:15.297889 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f72bb-1774-45db-a55e-c40bd0094827 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:15.298436 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f72bb-1774-45db-a55e-c40bd0094827 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:15.298436 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f72bb-1774-45db-a55e-c40bd0094827 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:15.299933 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 24/47] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:15.305920 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e69aa6-1f65-4cf8-a600-c84c4c34a602 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:15.306472 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e69aa6-1f65-4cf8-a600-c84c4c34a602 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:15.306562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e69aa6-1f65-4cf8-a600-c84c4c34a602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:15.324934 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-59e69aa6-1f65-4cf8-a600-c84c4c34a602 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:15.326529 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-59e69aa6-1f65-4cf8-a600-c84c4c34a602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:15.344581 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:15.394232 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 24/48] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:15.416759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f72bb-1774-45db-a55e-c40bd0094827 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:15.516834 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f72bb-1774-45db-a55e-c40bd0094827 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:15.549427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250c3c0-c61c-4982-916d-d26272f416e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:15.550224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250c3c0-c61c-4982-916d-d26272f416e4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:15.550512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250c3c0-c61c-4982-916d-d26272f416e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:15.550840 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250c3c0-c61c-4982-916d-d26272f416e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:15.562308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250c3c0-c61c-4982-916d-d26272f416e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:15.571082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-3250c3c0-c61c-4982-916d-d26272f416e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:24:15.571082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:15.571082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:15.571082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:15.571082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:15.571082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:15.571082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:15.571082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:15.571082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:15.571082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:15.571082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:15.571082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:15.571082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:15.571082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:15.571082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:15.571082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:15.571880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:15.571880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:15.571880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:15.571880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:15.571880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:15.571880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:15.571880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:15.571880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:15.571880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:15.571880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:15.571880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:15.571880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:24:15.571880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:15.571880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:15.571880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:15.571880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:15.572650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:15.572650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:15.572650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:15.572650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:15.572650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:15.572650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:15.572650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:15.572650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:15.572650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:15.572650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:15.572650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:24:15.572650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:24:15.572650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 25/49] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:24:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:15.578339 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e69aa6-1f65-4cf8-a600-c84c4c34a602 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:15.634376 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526a845-1a05-40e6-a026-9e0228eec2dc None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:15.634698 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526a845-1a05-40e6-a026-9e0228eec2dc None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:15.634991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526a845-1a05-40e6-a026-9e0228eec2dc None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:15.636361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 26/50] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 31 14:24:15.641470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c54b0-bc95-4331-8a68-253a542f979e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:15.641760 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c54b0-bc95-4331-8a68-253a542f979e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:15.642042 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c54b0-bc95-4331-8a68-253a542f979e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:15.658965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c38c54b0-bc95-4331-8a68-253a542f979e None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:15.660109 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c38c54b0-bc95-4331-8a68-253a542f979e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:15.676841 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e69aa6-1f65-4cf8-a600-c84c4c34a602 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:15.710028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd44f96-7ac1-4fea-a52f-6c6b20ab5523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:15.710745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd44f96-7ac1-4fea-a52f-6c6b20ab5523 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:15.711036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd44f96-7ac1-4fea-a52f-6c6b20ab5523 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:15.711315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd44f96-7ac1-4fea-a52f-6c6b20ab5523 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:15.712065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd44f96-7ac1-4fea-a52f-6c6b20ab5523 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:15.712386 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd44f96-7ac1-4fea-a52f-6c6b20ab5523 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:15.712554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:15.713790 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd44f96-7ac1-4fea-a52f-6c6b20ab5523 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:15.722302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 25/51] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:15.726843 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 27/52] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 31 14:24:15.727706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd44f96-7ac1-4fea-a52f-6c6b20ab5523 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:15.745458 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:15.750488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c54b0-bc95-4331-8a68-253a542f979e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:15.825943 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd44f96-7ac1-4fea-a52f-6c6b20ab5523 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:15.846735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c54b0-bc95-4331-8a68-253a542f979e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:15.859932 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889ed58-3297-44c6-b4a0-c12befce40d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:15.860811 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889ed58-3297-44c6-b4a0-c12befce40d1 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:15.861096 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889ed58-3297-44c6-b4a0-c12befce40d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:15.861374 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889ed58-3297-44c6-b4a0-c12befce40d1 None admin] PATH_INFO: `/v3/users/c01eab2db7454d578ce76aa7d022d460` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:15.862906 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889ed58-3297-44c6-b4a0-c12befce40d1 None admin] RBAC: Authorizing `identity:get_user(user_id=c01eab2db7454d578ce76aa7d022d460)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:15.864827 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889ed58-3297-44c6-b4a0-c12befce40d1 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:15.866851 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 26/53] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:15 2022] GET /identity/v3/users/c01eab2db7454d578ce76aa7d022d460 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:24:15.873435 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889ed58-3297-44c6-b4a0-c12befce40d1 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:15.886867 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87bf399-90a3-464d-916d-99522b2f8ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:15.887894 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bf399-90a3-464d-916d-99522b2f8ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:15.888084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bf399-90a3-464d-916d-99522b2f8ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:15.888303 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bf399-90a3-464d-916d-99522b2f8ae4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:15.889542 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87bf399-90a3-464d-916d-99522b2f8ae4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:15.891039 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87bf399-90a3-464d-916d-99522b2f8ae4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:15.892666 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 28/54] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:24:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:24:15.898117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87bf399-90a3-464d-916d-99522b2f8ae4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:15.953797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:15.963809 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:15.980466 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889ed58-3297-44c6-b4a0-c12befce40d1 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:15.997996 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87bf399-90a3-464d-916d-99522b2f8ae4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:16.014607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5a5cf-48f7-4ceb-85af-765acfc9b148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:16.015422 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5a5cf-48f7-4ceb-85af-765acfc9b148 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:16.015705 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5a5cf-48f7-4ceb-85af-765acfc9b148 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:16.015983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5a5cf-48f7-4ceb-85af-765acfc9b148 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:16.017159 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a5a5cf-48f7-4ceb-85af-765acfc9b148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:16.018606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a5a5cf-48f7-4ceb-85af-765acfc9b148 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:16.020466 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 27/55] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:24:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:16.025890 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5a5cf-48f7-4ceb-85af-765acfc9b148 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:16.031406 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3360d1-177e-4d8d-b437-1365c6348d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:16.032067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3360d1-177e-4d8d-b437-1365c6348d31 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:16.032354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3360d1-177e-4d8d-b437-1365c6348d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:16.032635 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3360d1-177e-4d8d-b437-1365c6348d31 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:16.033353 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3360d1-177e-4d8d-b437-1365c6348d31 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:16.034824 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3360d1-177e-4d8d-b437-1365c6348d31 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:16.059867 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 29/56] 10.222.0.67 () {60 vars in 1140 bytes} [Tue May 31 14:24:15 2022] POST /identity/v3/projects => generated 298 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:24:16.133632 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5a5cf-48f7-4ceb-85af-765acfc9b148 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:16.169949 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71e45c-b86e-428d-b104-fbdff317fc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:16.170623 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71e45c-b86e-428d-b104-fbdff317fc3f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:16.170941 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71e45c-b86e-428d-b104-fbdff317fc3f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:16.171222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71e45c-b86e-428d-b104-fbdff317fc3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:16.172072 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71e45c-b86e-428d-b104-fbdff317fc3f None admin] RBAC: Adding query filter params (role.id=e8d679596ffb45939229d1e2a13ce9b0, scope.domain.id=default, user.id=c01eab2db7454d578ce76aa7d022d460) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:16.172388 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71e45c-b86e-428d-b104-fbdff317fc3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:16.173859 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71e45c-b86e-428d-b104-fbdff317fc3f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:16.180950 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 28/57] 10.222.0.67 () {56 vars in 1322 bytes} [Tue May 31 14:24:16 2022] GET /identity/v3/role_assignments?user.id=c01eab2db7454d578ce76aa7d022d460&scope.domain.id=default&role.id=e8d679596ffb45939229d1e2a13ce9b0 => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:24:18.323936 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c57927-a82f-4fde-b007-4c25f66bb57d None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:18.324439 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c57927-a82f-4fde-b007-4c25f66bb57d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:18.324524 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c57927-a82f-4fde-b007-4c25f66bb57d None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:18.325885 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 30/58] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:18.331522 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0552c-26cf-4261-9dac-6cfa66e167a3 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:18.331887 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0552c-26cf-4261-9dac-6cfa66e167a3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:18.332090 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0552c-26cf-4261-9dac-6cfa66e167a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:18.348502 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-20c0552c-26cf-4261-9dac-6cfa66e167a3 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:18.349712 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-20c0552c-26cf-4261-9dac-6cfa66e167a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:18.360536 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:18.414922 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 29/59] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:18.439812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c57927-a82f-4fde-b007-4c25f66bb57d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:18.540220 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c57927-a82f-4fde-b007-4c25f66bb57d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:18.553295 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:18.585632 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97d83d-c325-46d0-a9a1-c72d133277cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:18.586430 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97d83d-c325-46d0-a9a1-c72d133277cc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:18.586753 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97d83d-c325-46d0-a9a1-c72d133277cc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:18.587030 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97d83d-c325-46d0-a9a1-c72d133277cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:18.588498 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97d83d-c325-46d0-a9a1-c72d133277cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:18.589980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97d83d-c325-46d0-a9a1-c72d133277cc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:18.591638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 31/60] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:24:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:18.597617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0552c-26cf-4261-9dac-6cfa66e167a3 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:18.692271 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0552c-26cf-4261-9dac-6cfa66e167a3 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:18.724866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9e938-351a-41af-afae-82e9221a8cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:18.725526 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9e938-351a-41af-afae-82e9221a8cf5 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:18.725818 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9e938-351a-41af-afae-82e9221a8cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:18.726100 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9e938-351a-41af-afae-82e9221a8cf5 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:18.726836 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9e938-351a-41af-afae-82e9221a8cf5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:18.728276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9e938-351a-41af-afae-82e9221a8cf5 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:18.752791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 30/61] 10.222.0.67 () {60 vars in 1132 bytes} [Tue May 31 14:24:18 2022] POST /identity/v3/users => generated 275 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:24:22.131544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8204253-4382-43bb-81e4-9dcd69bc9cd2 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:22.132357 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8204253-4382-43bb-81e4-9dcd69bc9cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:22.132826 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8204253-4382-43bb-81e4-9dcd69bc9cd2 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:22.134481 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 32/62] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:22.141648 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bec444-c569-4615-a364-98a919c53bb1 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:22.142143 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bec444-c569-4615-a364-98a919c53bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:22.142216 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bec444-c569-4615-a364-98a919c53bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:22.150739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:22.169484 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-66bec444-c569-4615-a364-98a919c53bb1 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:22.171128 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-66bec444-c569-4615-a364-98a919c53bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:22.180482 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a0f63-5f9d-4910-b479-96450ea08e65 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:22.181142 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a0f63-5f9d-4910-b479-96450ea08e65 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:22.181624 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a0f63-5f9d-4910-b479-96450ea08e65 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:22.183352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 33/63] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 31 14:24:22.189037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a643a5-2dde-465b-8fd6-36e20383466e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:22.189655 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a643a5-2dde-465b-8fd6-36e20383466e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:22.190143 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a643a5-2dde-465b-8fd6-36e20383466e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:22.225986 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b6a643a5-2dde-465b-8fd6-36e20383466e None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:22.227234 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b6a643a5-2dde-465b-8fd6-36e20383466e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:22.307038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 31/64] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:22.352763 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bec444-c569-4615-a364-98a919c53bb1 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:22.374160 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 34/65] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:22.397482 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a643a5-2dde-465b-8fd6-36e20383466e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:22.524977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bec444-c569-4615-a364-98a919c53bb1 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:22.562250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:22.568904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a643a5-2dde-465b-8fd6-36e20383466e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:22.569236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a638f39-e039-4663-b299-7b2c2fab462f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:22.570246 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a638f39-e039-4663-b299-7b2c2fab462f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:22.570908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a638f39-e039-4663-b299-7b2c2fab462f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:22.572200 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a638f39-e039-4663-b299-7b2c2fab462f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:22.578178 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:22.586725 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a638f39-e039-4663-b299-7b2c2fab462f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:22.598456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-8a638f39-e039-4663-b299-7b2c2fab462f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 31 14:24:22.598456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:22.598456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:22.598456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:22.598456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:22.598456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:22.598456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:22.598456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:22.598456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:22.598456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:22.598456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:22.598456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:22.598456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:22.598456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:22.598456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:22.598456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:22.599389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:22.599389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:22.599389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:22.599389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:22.599389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:22.599389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:22.599389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:22.599389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:22.599389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:22.599389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:22.599389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:22.599389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:24:22.599389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:22.599389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:22.599389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:22.599389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:22.601276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:22.601276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:22.601276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:22.601276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:22.601276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:22.601276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:22.601276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:22.601276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:22.601276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:22.601276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:22.601276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 31 14:24:22.601276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:24:22.603970 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 32/66] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:24:22.605898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05d152-ad6f-47c5-b624-0ada06b69777 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:22.606333 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05d152-ad6f-47c5-b624-0ada06b69777 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:22.607110 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05d152-ad6f-47c5-b624-0ada06b69777 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:22.608732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 33/67] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:22.613481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05d152-ad6f-47c5-b624-0ada06b69777 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:22.615980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1003d83-eb32-42f2-be39-b2ebbccf5006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:22.617209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1003d83-eb32-42f2-be39-b2ebbccf5006 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:22.618525 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1003d83-eb32-42f2-be39-b2ebbccf5006 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:22.618944 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1003d83-eb32-42f2-be39-b2ebbccf5006 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:22.633177 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1003d83-eb32-42f2-be39-b2ebbccf5006 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:22.644221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-c1003d83-eb32-42f2-be39-b2ebbccf5006 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:24:22.644221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:22.644221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:22.644221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:22.644221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:22.644221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:22.644221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:22.644221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:22.644221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:22.644221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:22.644221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:22.644221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:22.644221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:22.644221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:22.644221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:22.644221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:22.645125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:22.645125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:22.645125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:22.645125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:22.645125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:22.645125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:22.645125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:22.645125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:22.645125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:22.645125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:22.645125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:22.645125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:24:22.645125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:22.645125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:22.645125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:22.645125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:22.645887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:22.645887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:22.645887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:22.645887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:22.645887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:22.645887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:22.645887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:22.645887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:22.645887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:22.645887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:22.645887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:24:22.645887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:24:22.645887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 35/68] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:24:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:24:22.647036 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce544b-ea20-4950-8bb1-8ab20baa791f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:22.647036 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce544b-ea20-4950-8bb1-8ab20baa791f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:22.647272 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce544b-ea20-4950-8bb1-8ab20baa791f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:22.668835 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-abce544b-ea20-4950-8bb1-8ab20baa791f None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:22.670277 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-abce544b-ea20-4950-8bb1-8ab20baa791f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:22.742044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 36/69] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:22.742921 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce544b-ea20-4950-8bb1-8ab20baa791f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:22.752368 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05d152-ad6f-47c5-b624-0ada06b69777 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:22.794609 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd3f8b-a899-44a9-bc4d-9dcfb9cf7219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:22.796172 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd3f8b-a899-44a9-bc4d-9dcfb9cf7219 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:22.796727 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd3f8b-a899-44a9-bc4d-9dcfb9cf7219 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:22.797019 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd3f8b-a899-44a9-bc4d-9dcfb9cf7219 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:22.797932 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd3f8b-a899-44a9-bc4d-9dcfb9cf7219 None admin] RBAC: Adding query filter params (name=member) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:22.798250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd3f8b-a899-44a9-bc4d-9dcfb9cf7219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:22.799977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd3f8b-a899-44a9-bc4d-9dcfb9cf7219 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:22.810096 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 34/70] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:24:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:22.812248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bc199-21a3-4943-a284-82dd6f77da65 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:22.812604 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bc199-21a3-4943-a284-82dd6f77da65 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:22.812954 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bc199-21a3-4943-a284-82dd6f77da65 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:22.814420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 35/71] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:22.815144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605bc199-21a3-4943-a284-82dd6f77da65 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:22.847388 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:22.853951 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce544b-ea20-4950-8bb1-8ab20baa791f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:22.871145 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:22.888033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28584e9-571d-46d5-bb31-03a6a22135aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:22.888740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28584e9-571d-46d5-bb31-03a6a22135aa None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:22.889035 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28584e9-571d-46d5-bb31-03a6a22135aa None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:22.889318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28584e9-571d-46d5-bb31-03a6a22135aa None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:22.890087 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28584e9-571d-46d5-bb31-03a6a22135aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:22.890401 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28584e9-571d-46d5-bb31-03a6a22135aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:22.891838 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28584e9-571d-46d5-bb31-03a6a22135aa None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:22.899873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 37/72] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:22.901053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65a514-4cca-49b2-a4ee-a594418d0a79 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:22.901349 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65a514-4cca-49b2-a4ee-a594418d0a79 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:22.901632 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65a514-4cca-49b2-a4ee-a594418d0a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:22.917118 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5f65a514-4cca-49b2-a4ee-a594418d0a79 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:22.918316 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5f65a514-4cca-49b2-a4ee-a594418d0a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:22.921868 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605bc199-21a3-4943-a284-82dd6f77da65 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:22.957214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4732859-dc3e-43aa-97d8-355be79b0ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:22.958027 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4732859-dc3e-43aa-97d8-355be79b0ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:22.958314 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4732859-dc3e-43aa-97d8-355be79b0ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:22.958594 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4732859-dc3e-43aa-97d8-355be79b0ae6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:22.968968 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4732859-dc3e-43aa-97d8-355be79b0ae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:22.977076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-e4732859-dc3e-43aa-97d8-355be79b0ae6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:24:22.977076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:22.977076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:22.977076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:22.977076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:22.977076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:22.977076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:22.977076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:22.977076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:22.977076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:22.977076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:22.977076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:22.977076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:22.977076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:22.977076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:22.977076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:22.978051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:22.978051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:22.978051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:22.978051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:22.978051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:22.978051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:22.978051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:22.978051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:22.978051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:22.978051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:22.978051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:22.978051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:24:22.978051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:22.978051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:22.978051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:22.978051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:22.978866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:22.978866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:22.978866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:22.978866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:22.978866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:22.978866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:22.978866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:22.978866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:22.978866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:22.978866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:22.978866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:24:22.978866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:24:22.978866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 36/73] 10.222.0.67 () {56 vars in 1114 bytes} [Tue May 31 14:24:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:22.979720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4732859-dc3e-43aa-97d8-355be79b0ae6 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:22.987348 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 38/74] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:22.988744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a61a37-969b-4156-a797-2b2f0e0fbe23 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:22.989046 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a61a37-969b-4156-a797-2b2f0e0fbe23 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:22.989322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a61a37-969b-4156-a797-2b2f0e0fbe23 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:22.990601 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 39/75] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:22.991322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a61a37-969b-4156-a797-2b2f0e0fbe23 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:23.082711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4732859-dc3e-43aa-97d8-355be79b0ae6 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:23.087176 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:23.096608 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:23.101370 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a61a37-969b-4156-a797-2b2f0e0fbe23 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:23.116989 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15a5b7-ce7f-4365-910c-c3ee73ab36bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:23.117878 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15a5b7-ce7f-4365-910c-c3ee73ab36bd None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:23.118165 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15a5b7-ce7f-4365-910c-c3ee73ab36bd None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:23.118443 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15a5b7-ce7f-4365-910c-c3ee73ab36bd None admin] PATH_INFO: `/v3/users/f89c7583bd684f1a883ca3ccb2230e5b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:23.133775 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b15a5b7-ce7f-4365-910c-c3ee73ab36bd None admin] RBAC: Authorizing `identity:get_user(user_id=f89c7583bd684f1a883ca3ccb2230e5b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:23.135295 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b15a5b7-ce7f-4365-910c-c3ee73ab36bd None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:23.136238 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90bbd6-269f-4467-b07a-d0c200173e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:23.136998 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 37/76] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:22 2022] GET /identity/v3/users/f89c7583bd684f1a883ca3ccb2230e5b => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:23.137127 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90bbd6-269f-4467-b07a-d0c200173e0d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:23.137399 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90bbd6-269f-4467-b07a-d0c200173e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:23.137678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90bbd6-269f-4467-b07a-d0c200173e0d None admin] PATH_INFO: `/v3/users/c01eab2db7454d578ce76aa7d022d460` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:23.137770 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15a5b7-ce7f-4365-910c-c3ee73ab36bd None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:23.139202 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90bbd6-269f-4467-b07a-d0c200173e0d None admin] RBAC: Authorizing `identity:get_user(user_id=c01eab2db7454d578ce76aa7d022d460)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:23.140640 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90bbd6-269f-4467-b07a-d0c200173e0d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:23.142538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 40/77] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:22 2022] GET /identity/v3/users/c01eab2db7454d578ce76aa7d022d460 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:23.143708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e67d5f-e217-4911-bf8c-edfc6836e3e2 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:23.143998 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e67d5f-e217-4911-bf8c-edfc6836e3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:23.144278 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e67d5f-e217-4911-bf8c-edfc6836e3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:23.160848 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b1e67d5f-e217-4911-bf8c-edfc6836e3e2 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:23.162053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b1e67d5f-e217-4911-bf8c-edfc6836e3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:23.227689 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 41/78] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:23.228420 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e67d5f-e217-4911-bf8c-edfc6836e3e2 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:23.236200 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15a5b7-ce7f-4365-910c-c3ee73ab36bd None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:23.273948 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a757a72-5c42-430f-a287-278bfaab5ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:23.274744 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a757a72-5c42-430f-a287-278bfaab5ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:23.275039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a757a72-5c42-430f-a287-278bfaab5ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:23.275324 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a757a72-5c42-430f-a287-278bfaab5ce1 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:23.276078 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a757a72-5c42-430f-a287-278bfaab5ce1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:23.276394 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a757a72-5c42-430f-a287-278bfaab5ce1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:23.277797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a757a72-5c42-430f-a287-278bfaab5ce1 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:23.286154 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 38/79] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:24:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:23.286984 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a757a72-5c42-430f-a287-278bfaab5ce1 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:23.320725 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:23.325504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:23.340633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e67d5f-e217-4911-bf8c-edfc6836e3e2 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:23.374655 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da583c-8f17-4ea3-9433-43a97ca4f40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:23.375490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da583c-8f17-4ea3-9433-43a97ca4f40d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:23.375783 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da583c-8f17-4ea3-9433-43a97ca4f40d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:23.376073 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da583c-8f17-4ea3-9433-43a97ca4f40d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:23.384854 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a757a72-5c42-430f-a287-278bfaab5ce1 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:23.386355 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da583c-8f17-4ea3-9433-43a97ca4f40d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:23.394055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-a9da583c-8f17-4ea3-9433-43a97ca4f40d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 31 14:24:23.394055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:23.394055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:23.394055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:23.394055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:23.394055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:23.394055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:23.394055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:23.394055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:23.394055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:23.394055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:23.394055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:23.394055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:23.394055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:23.394055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:23.394055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:23.394967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:23.394967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:23.394967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:23.394967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:23.394967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:23.394967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:23.394967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:23.394967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:23.394967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:23.394967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:23.394967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:23.394967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:24:23.394967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:23.394967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:23.394967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:23.394967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:23.395745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:23.395745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:23.395745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:23.395745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:23.395745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:23.395745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:23.395745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:23.395745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:23.395745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:23.395745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:23.395745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 31 14:24:23.395745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:24:23.395745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 42/80] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:23.396371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da583c-8f17-4ea3-9433-43a97ca4f40d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:23.423309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f93a36-0d34-4d4e-be04-b4006312c606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:23.423959 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f93a36-0d34-4d4e-be04-b4006312c606 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:23.424246 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f93a36-0d34-4d4e-be04-b4006312c606 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:23.424529 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f93a36-0d34-4d4e-be04-b4006312c606 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:23.491563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f93a36-0d34-4d4e-be04-b4006312c606 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:23.492969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f93a36-0d34-4d4e-be04-b4006312c606 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:23.498729 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da583c-8f17-4ea3-9433-43a97ca4f40d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:23.524976 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:23.533462 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7d7af-b5e6-4a2c-8fbd-31799af4e266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:23.534108 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7d7af-b5e6-4a2c-8fbd-31799af4e266 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:23.534401 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7d7af-b5e6-4a2c-8fbd-31799af4e266 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:23.534713 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7d7af-b5e6-4a2c-8fbd-31799af4e266 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:23.549519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:23.590282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 39/81] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:24:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:24:23.591145 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f93a36-0d34-4d4e-be04-b4006312c606 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:23.603342 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7d7af-b5e6-4a2c-8fbd-31799af4e266 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:23.604819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7d7af-b5e6-4a2c-8fbd-31799af4e266 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:23.688669 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f93a36-0d34-4d4e-be04-b4006312c606 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:23.701819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 43/82] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:24:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:24:23.702945 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7d7af-b5e6-4a2c-8fbd-31799af4e266 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:23.724284 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1369e-43a5-4c15-b19e-f75577bec06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:23.725079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1369e-43a5-4c15-b19e-f75577bec06f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:23.725364 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1369e-43a5-4c15-b19e-f75577bec06f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:23.725639 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1369e-43a5-4c15-b19e-f75577bec06f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:23.726820 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1369e-43a5-4c15-b19e-f75577bec06f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:23.728278 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1369e-43a5-4c15-b19e-f75577bec06f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:23.730062 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 40/83] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:24:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:23.730728 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1369e-43a5-4c15-b19e-f75577bec06f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:23.733334 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:23.753981 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:23.806375 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7d7af-b5e6-4a2c-8fbd-31799af4e266 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:23.830179 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1369e-43a5-4c15-b19e-f75577bec06f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:23.840339 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700479d-50e4-471d-b491-2e53b9afc17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:23.841207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700479d-50e4-471d-b491-2e53b9afc17d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:23.841491 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700479d-50e4-471d-b491-2e53b9afc17d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:23.842345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700479d-50e4-471d-b491-2e53b9afc17d None admin] PATH_INFO: `/v3/users/f89c7583bd684f1a883ca3ccb2230e5b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:23.843570 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700479d-50e4-471d-b491-2e53b9afc17d None admin] RBAC: Authorizing `identity:get_user(user_id=f89c7583bd684f1a883ca3ccb2230e5b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:23.845015 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700479d-50e4-471d-b491-2e53b9afc17d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:23.846671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 44/84] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:23 2022] GET /identity/v3/users/f89c7583bd684f1a883ca3ccb2230e5b => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:23.847425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700479d-50e4-471d-b491-2e53b9afc17d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:23.864883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fd69f-385b-42e3-9cb2-33638ea8c6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:23.865581 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fd69f-385b-42e3-9cb2-33638ea8c6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:23.865864 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fd69f-385b-42e3-9cb2-33638ea8c6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:23.866141 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fd69f-385b-42e3-9cb2-33638ea8c6f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:23.866914 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580fd69f-385b-42e3-9cb2-33638ea8c6f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:23.867223 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580fd69f-385b-42e3-9cb2-33638ea8c6f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:23.868614 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580fd69f-385b-42e3-9cb2-33638ea8c6f8 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:23.876962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 41/85] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:24:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:23.877729 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fd69f-385b-42e3-9cb2-33638ea8c6f8 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:23.937747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:23.949067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700479d-50e4-471d-b491-2e53b9afc17d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:23.963699 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:23.976013 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fd69f-385b-42e3-9cb2-33638ea8c6f8 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:23.984004 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6872c9-0e4e-4e32-aea5-055a921fafbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:23.984867 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6872c9-0e4e-4e32-aea5-055a921fafbb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:23.985156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6872c9-0e4e-4e32-aea5-055a921fafbb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:23.985445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6872c9-0e4e-4e32-aea5-055a921fafbb None admin] PATH_INFO: `/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:23.986664 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6872c9-0e4e-4e32-aea5-055a921fafbb None admin] RBAC: Authorizing `identity:get_project(project_id=e3f26f512b354f6ebdbf6c8d3cc3e472)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:23.988116 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6872c9-0e4e-4e32-aea5-055a921fafbb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:23.989819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 45/86] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:23 2022] GET /identity/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:23.990609 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6872c9-0e4e-4e32-aea5-055a921fafbb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:24.011308 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18358f-cd78-46cf-80b7-4792b2e6bccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:24.012101 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18358f-cd78-46cf-80b7-4792b2e6bccb None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:24.012384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18358f-cd78-46cf-80b7-4792b2e6bccb None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:24.012659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18358f-cd78-46cf-80b7-4792b2e6bccb None admin] PATH_INFO: `/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:24.014367 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18358f-cd78-46cf-80b7-4792b2e6bccb None admin] RBAC: Authorizing `identity:get_project(project_id=e3f26f512b354f6ebdbf6c8d3cc3e472)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:24.015854 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18358f-cd78-46cf-80b7-4792b2e6bccb None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:24.017539 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 42/87] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:23 2022] GET /identity/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:24.018352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18358f-cd78-46cf-80b7-4792b2e6bccb None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:24.092110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6872c9-0e4e-4e32-aea5-055a921fafbb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:24.115842 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18358f-cd78-46cf-80b7-4792b2e6bccb None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:24.127482 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d468ec-4ba8-464d-99fc-b3e5147785b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:24.128158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d468ec-4ba8-464d-99fc-b3e5147785b4 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:24.128442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d468ec-4ba8-464d-99fc-b3e5147785b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:24.128716 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d468ec-4ba8-464d-99fc-b3e5147785b4 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:24.129722 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d468ec-4ba8-464d-99fc-b3e5147785b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:24.130902 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d468ec-4ba8-464d-99fc-b3e5147785b4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:24.147586 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:24.153926 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cd239-b913-4af7-acbe-5ce76ea8d9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:24.154718 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cd239-b913-4af7-acbe-5ce76ea8d9ef None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:24.155009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cd239-b913-4af7-acbe-5ce76ea8d9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:24.155291 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cd239-b913-4af7-acbe-5ce76ea8d9ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:24.156969 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 46/88] 10.222.0.67 () {60 vars in 1132 bytes} [Tue May 31 14:24:23 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:24:24.157784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d468ec-4ba8-464d-99fc-b3e5147785b4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:24.227600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:24.229936 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329cd239-b913-4af7-acbe-5ce76ea8d9ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:24.231765 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329cd239-b913-4af7-acbe-5ce76ea8d9ef None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:24.299433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d468ec-4ba8-464d-99fc-b3e5147785b4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:24.336896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2331f0-fc1f-478e-aed1-efe13a98e885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:24.337804 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2331f0-fc1f-478e-aed1-efe13a98e885 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:24.338098 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2331f0-fc1f-478e-aed1-efe13a98e885 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:24.338384 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2331f0-fc1f-478e-aed1-efe13a98e885 None admin] PATH_INFO: `/v3/users/f89c7583bd684f1a883ca3ccb2230e5b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:24.339617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 43/89] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:24:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:24:24.340059 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2331f0-fc1f-478e-aed1-efe13a98e885 None admin] RBAC: Authorizing `identity:get_user(user_id=f89c7583bd684f1a883ca3ccb2230e5b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:24.341043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cd239-b913-4af7-acbe-5ce76ea8d9ef None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:24.341143 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2331f0-fc1f-478e-aed1-efe13a98e885 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:24.347154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 47/90] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:24 2022] GET /identity/v3/users/f89c7583bd684f1a883ca3ccb2230e5b => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:24.348039 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2331f0-fc1f-478e-aed1-efe13a98e885 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:24.377799 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:24.469384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cd239-b913-4af7-acbe-5ce76ea8d9ef None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:24.473487 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2331f0-fc1f-478e-aed1-efe13a98e885 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:24.483761 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:24.514454 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87875aff-a7f5-4600-b196-6309df7d4fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:24.515153 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87875aff-a7f5-4600-b196-6309df7d4fac None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:24.515445 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87875aff-a7f5-4600-b196-6309df7d4fac None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:24.515601 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68e1ad-5b34-4709-939b-787b8ed09f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:24.515767 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87875aff-a7f5-4600-b196-6309df7d4fac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:24.516233 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68e1ad-5b34-4709-939b-787b8ed09f35 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:24.516520 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68e1ad-5b34-4709-939b-787b8ed09f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:24.516606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87875aff-a7f5-4600-b196-6309df7d4fac None admin] RBAC: Adding query filter params (role.id=9ce8fdecf52d482d894fdeec4f530624, scope.project.id=e3f26f512b354f6ebdbf6c8d3cc3e472, user.id=f89c7583bd684f1a883ca3ccb2230e5b) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:24.516799 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68e1ad-5b34-4709-939b-787b8ed09f35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:24.516888 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87875aff-a7f5-4600-b196-6309df7d4fac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:24.517631 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68e1ad-5b34-4709-939b-787b8ed09f35 None admin] RBAC: Adding query filter params (role.id=e8d679596ffb45939229d1e2a13ce9b0, scope.project.id=e3f26f512b354f6ebdbf6c8d3cc3e472, user.id=c01eab2db7454d578ce76aa7d022d460) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:24.517944 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68e1ad-5b34-4709-939b-787b8ed09f35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:24.518326 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87875aff-a7f5-4600-b196-6309df7d4fac None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:24.519429 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68e1ad-5b34-4709-939b-787b8ed09f35 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:24.524215 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 44/91] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:24:24 2022] GET /identity/v3/role_assignments?user.id=f89c7583bd684f1a883ca3ccb2230e5b&scope.project.id=e3f26f512b354f6ebdbf6c8d3cc3e472&role.id=9ce8fdecf52d482d894fdeec4f530624 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:24.525140 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87875aff-a7f5-4600-b196-6309df7d4fac None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:24.528408 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 48/92] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:24:24 2022] GET /identity/v3/role_assignments?user.id=c01eab2db7454d578ce76aa7d022d460&scope.project.id=e3f26f512b354f6ebdbf6c8d3cc3e472&role.id=e8d679596ffb45939229d1e2a13ce9b0 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:24.530332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68e1ad-5b34-4709-939b-787b8ed09f35 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:24.646118 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68e1ad-5b34-4709-939b-787b8ed09f35 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:24.685597 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a8c60-cdea-4c54-8795-e02cbe693dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:24.690683 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a8c60-cdea-4c54-8795-e02cbe693dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:24.690683 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a8c60-cdea-4c54-8795-e02cbe693dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:24.690683 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a8c60-cdea-4c54-8795-e02cbe693dc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:24.725519 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:24.739439 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87875aff-a7f5-4600-b196-6309df7d4fac None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:24.775040 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:24.814401 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ae56f-ed8a-4841-89ef-9770aac09234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:24.815587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ae56f-ed8a-4841-89ef-9770aac09234 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:24.815886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ae56f-ed8a-4841-89ef-9770aac09234 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:24.816171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ae56f-ed8a-4841-89ef-9770aac09234 None admin] PATH_INFO: `/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:24.818065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ae56f-ed8a-4841-89ef-9770aac09234 None admin] RBAC: Authorizing `identity:get_project(project_id=e3f26f512b354f6ebdbf6c8d3cc3e472)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:24.819824 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ae56f-ed8a-4841-89ef-9770aac09234 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:24.822737 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 45/93] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:24 2022] GET /identity/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472 => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:24.828775 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ae56f-ed8a-4841-89ef-9770aac09234 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:24.829201 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a8c60-cdea-4c54-8795-e02cbe693dc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:24.830658 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a8c60-cdea-4c54-8795-e02cbe693dc2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:24.939707 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 49/94] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:24:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:24:24.946686 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a8c60-cdea-4c54-8795-e02cbe693dc2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:25.003496 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ae56f-ed8a-4841-89ef-9770aac09234 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:25.021019 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:25.054210 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a8c60-cdea-4c54-8795-e02cbe693dc2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:25.069964 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf5d46-e470-4a12-8883-c76b108b2cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:25.070723 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf5d46-e470-4a12-8883-c76b108b2cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:25.071022 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf5d46-e470-4a12-8883-c76b108b2cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:25.071307 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf5d46-e470-4a12-8883-c76b108b2cf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:25.072361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf5d46-e470-4a12-8883-c76b108b2cf7 None admin] RBAC: Adding query filter params (role.id=1b9c18d4b8b540bf9825080748cec3bc, scope.project.id=e3f26f512b354f6ebdbf6c8d3cc3e472, user.id=f89c7583bd684f1a883ca3ccb2230e5b) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:25.072690 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf5d46-e470-4a12-8883-c76b108b2cf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:25.074160 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf5d46-e470-4a12-8883-c76b108b2cf7 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:25.088913 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 46/95] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:24:24 2022] GET /identity/v3/role_assignments?user.id=f89c7583bd684f1a883ca3ccb2230e5b&scope.project.id=e3f26f512b354f6ebdbf6c8d3cc3e472&role.id=1b9c18d4b8b540bf9825080748cec3bc => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:25.090174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e3c50-e04b-4304-b491-356a90a60037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:25.091089 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e3c50-e04b-4304-b491-356a90a60037 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:25.091378 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e3c50-e04b-4304-b491-356a90a60037 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:25.092282 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e3c50-e04b-4304-b491-356a90a60037 None admin] PATH_INFO: `/v3/projects/1aac711712614ffd98b5667d5dc8bc91` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:25.093531 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e3c50-e04b-4304-b491-356a90a60037 None admin] RBAC: Authorizing `identity:get_project(project_id=1aac711712614ffd98b5667d5dc8bc91)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:25.095053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e3c50-e04b-4304-b491-356a90a60037 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:25.097219 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 50/96] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:24 2022] GET /identity/v3/projects/1aac711712614ffd98b5667d5dc8bc91 => generated 312 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:25.102957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf5d46-e470-4a12-8883-c76b108b2cf7 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:25.166689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:25.246479 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf5d46-e470-4a12-8883-c76b108b2cf7 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:25.293153 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0a5b4-a2d9-4e73-8048-2a3baad9df9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:25.293888 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0a5b4-a2d9-4e73-8048-2a3baad9df9a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:25.294180 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0a5b4-a2d9-4e73-8048-2a3baad9df9a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:25.294462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0a5b4-a2d9-4e73-8048-2a3baad9df9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:25.295390 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0a5b4-a2d9-4e73-8048-2a3baad9df9a None admin] RBAC: Adding query filter params (role.id=9ce8fdecf52d482d894fdeec4f530624, scope.project.id=1aac711712614ffd98b5667d5dc8bc91, user.id=f89c7583bd684f1a883ca3ccb2230e5b) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:25.295711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0a5b4-a2d9-4e73-8048-2a3baad9df9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:25.297222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0a5b4-a2d9-4e73-8048-2a3baad9df9a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:25.307640 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 47/97] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:24:25 2022] GET /identity/v3/role_assignments?user.id=f89c7583bd684f1a883ca3ccb2230e5b&scope.project.id=1aac711712614ffd98b5667d5dc8bc91&role.id=9ce8fdecf52d482d894fdeec4f530624 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:27.430544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba210590-6f56-4eee-af04-b09e86363eae None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:27.431102 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba210590-6f56-4eee-af04-b09e86363eae None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:27.431102 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba210590-6f56-4eee-af04-b09e86363eae None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:27.432735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 51/98] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:27.437891 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c742e-a415-4f1d-9835-8b863cd9184c None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:27.438295 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c742e-a415-4f1d-9835-8b863cd9184c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:27.438473 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c742e-a415-4f1d-9835-8b863cd9184c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:27.460021 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-974c742e-a415-4f1d-9835-8b863cd9184c None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:27.461205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-974c742e-a415-4f1d-9835-8b863cd9184c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:27.548167 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 48/99] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:27.570929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba210590-6f56-4eee-af04-b09e86363eae None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:27.622116 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5357b3-4c0d-45fc-9852-fd3c3e752b31 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:27.622460 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5357b3-4c0d-45fc-9852-fd3c3e752b31 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:27.623202 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5357b3-4c0d-45fc-9852-fd3c3e752b31 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:27.625275 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 49/100] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 31 14:24:27.630176 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba3301-3300-4635-9dc2-13b282fce186 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:27.630468 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba3301-3300-4635-9dc2-13b282fce186 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:27.630810 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba3301-3300-4635-9dc2-13b282fce186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:27.654617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-15ba3301-3300-4635-9dc2-13b282fce186 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:27.654617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-15ba3301-3300-4635-9dc2-13b282fce186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:27.692590 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:27.699924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba210590-6f56-4eee-af04-b09e86363eae None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:27.772453 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8aca4-565f-4bb2-a0b4-62e57a98e719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:27.773272 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8aca4-565f-4bb2-a0b4-62e57a98e719 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:27.773560 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8aca4-565f-4bb2-a0b4-62e57a98e719 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:27.773838 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8aca4-565f-4bb2-a0b4-62e57a98e719 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:27.786748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:27.792362 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf8aca4-565f-4bb2-a0b4-62e57a98e719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:27.809493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-0bf8aca4-565f-4bb2-a0b4-62e57a98e719 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 31 14:24:27.809493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:27.809493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:27.809493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:27.809493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:27.809493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:27.809493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:27.809493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:27.809493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:27.809493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:27.809493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:27.809493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:27.809493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:27.809493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:27.809493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:27.809493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:27.811002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:27.811002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:27.811002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:27.811002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:27.811002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:27.811002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:27.811002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:27.811002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:27.811002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:27.811002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:27.811002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:27.811002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:24:27.811002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:27.811002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:27.811002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:27.811002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:27.811805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:27.811805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:27.811805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:27.811805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:27.811805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:27.811805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:27.811805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:27.811805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:27.811805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:27.811805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:27.811805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 31 14:24:27.811805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:24:27.812918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 50/101] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:27.814702 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 52/102] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:27.828595 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba3301-3300-4635-9dc2-13b282fce186 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:27.880611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8aca4-565f-4bb2-a0b4-62e57a98e719 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:28.029072 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba3301-3300-4635-9dc2-13b282fce186 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:28.107601 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac4b75-b0ed-4e9b-96bf-99a8f4766c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:28.108428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac4b75-b0ed-4e9b-96bf-99a8f4766c0b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:28.108720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac4b75-b0ed-4e9b-96bf-99a8f4766c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:28.109002 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac4b75-b0ed-4e9b-96bf-99a8f4766c0b None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:28.109787 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac4b75-b0ed-4e9b-96bf-99a8f4766c0b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:28.110140 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac4b75-b0ed-4e9b-96bf-99a8f4766c0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:28.111591 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac4b75-b0ed-4e9b-96bf-99a8f4766c0b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:28.125304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8aca4-565f-4bb2-a0b4-62e57a98e719 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:28.139749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 51/103] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:24:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:28.145208 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac4b75-b0ed-4e9b-96bf-99a8f4766c0b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:28.179513 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:28.209082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8977c0-bac6-44f7-8b16-8d0f47fa5ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:28.209924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8977c0-bac6-44f7-8b16-8d0f47fa5ebb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:28.210218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8977c0-bac6-44f7-8b16-8d0f47fa5ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:28.210567 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8977c0-bac6-44f7-8b16-8d0f47fa5ebb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:28.211953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8977c0-bac6-44f7-8b16-8d0f47fa5ebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:28.213837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8977c0-bac6-44f7-8b16-8d0f47fa5ebb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:28.215542 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 53/104] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:24:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:28.216808 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a615b1-e581-444a-8bce-51ab81973722 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:28.217104 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a615b1-e581-444a-8bce-51ab81973722 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:28.217388 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a615b1-e581-444a-8bce-51ab81973722 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:28.218786 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 54/105] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:28.228356 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a615b1-e581-444a-8bce-51ab81973722 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:28.346536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:28.353324 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac4b75-b0ed-4e9b-96bf-99a8f4766c0b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:28.414550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c546c78-4a0a-44ee-a637-8cb753dd96f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:28.416525 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c546c78-4a0a-44ee-a637-8cb753dd96f1 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:28.416525 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c546c78-4a0a-44ee-a637-8cb753dd96f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:28.416525 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c546c78-4a0a-44ee-a637-8cb753dd96f1 None admin] PATH_INFO: `/v3/users/2d38ae56e517482e86864dab992d1c22` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:28.420196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a615b1-e581-444a-8bce-51ab81973722 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:28.451575 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c546c78-4a0a-44ee-a637-8cb753dd96f1 None admin] RBAC: Authorizing `identity:get_user(user_id=2d38ae56e517482e86864dab992d1c22)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:28.453344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c546c78-4a0a-44ee-a637-8cb753dd96f1 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:28.459068 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 52/106] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:28 2022] GET /identity/v3/users/2d38ae56e517482e86864dab992d1c22 => generated 289 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:28.459068 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654e84d-f69a-4d42-81d2-bb890108aec5 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:28.459068 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654e84d-f69a-4d42-81d2-bb890108aec5 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:28.459068 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654e84d-f69a-4d42-81d2-bb890108aec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:28.473143 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a248ea48-cf02-4f80-87c3-7bbe19349d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:28.473953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248ea48-cf02-4f80-87c3-7bbe19349d82 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:28.474347 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248ea48-cf02-4f80-87c3-7bbe19349d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:28.474760 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248ea48-cf02-4f80-87c3-7bbe19349d82 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:28.475884 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a248ea48-cf02-4f80-87c3-7bbe19349d82 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:28.477425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a248ea48-cf02-4f80-87c3-7bbe19349d82 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:28.479690 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-8654e84d-f69a-4d42-81d2-bb890108aec5 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:28.481209 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8654e84d-f69a-4d42-81d2-bb890108aec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:28.514672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 55/107] 10.222.0.67 () {60 vars in 1141 bytes} [Tue May 31 14:24:28 2022] POST /identity/v3/projects => generated 302 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:24:28.514672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067aa0c-6f0b-4ba5-a05b-f7adec251bc3 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:28.514672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067aa0c-6f0b-4ba5-a05b-f7adec251bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:28.514672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067aa0c-6f0b-4ba5-a05b-f7adec251bc3 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:28.514672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 56/108] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:28.514672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067aa0c-6f0b-4ba5-a05b-f7adec251bc3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:28.588629 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 53/109] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:28.590696 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c9afd-4e38-4c75-9318-ff754287df7d None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:28.591178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c9afd-4e38-4c75-9318-ff754287df7d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:28.594261 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c9afd-4e38-4c75-9318-ff754287df7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:28.604426 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:28.625378 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-de1c9afd-4e38-4c75-9318-ff754287df7d None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:28.627120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-de1c9afd-4e38-4c75-9318-ff754287df7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:28.652569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067aa0c-6f0b-4ba5-a05b-f7adec251bc3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:28.675526 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:28.696956 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d6432-4d20-4568-b4db-9f88911a6f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:28.697639 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d6432-4d20-4568-b4db-9f88911a6f50 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:28.698087 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d6432-4d20-4568-b4db-9f88911a6f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:28.698372 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d6432-4d20-4568-b4db-9f88911a6f50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:28.713772 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 54/110] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:28.720703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8adada-0a00-47de-bf37-8940f28e78d6 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:28.721294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8adada-0a00-47de-bf37-8940f28e78d6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:28.721828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8adada-0a00-47de-bf37-8940f28e78d6 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:28.724571 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 55/111] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:28 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:28.726001 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8adada-0a00-47de-bf37-8940f28e78d6 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:28.780445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d6432-4d20-4568-b4db-9f88911a6f50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:28.781945 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d6432-4d20-4568-b4db-9f88911a6f50 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:28.847507 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8adada-0a00-47de-bf37-8940f28e78d6 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:28.876384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:28.884385 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53eb85f-9f06-447a-aeb7-e73f92feb172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:28.885218 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53eb85f-9f06-447a-aeb7-e73f92feb172 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:28.885510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53eb85f-9f06-447a-aeb7-e73f92feb172 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:28.885791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53eb85f-9f06-447a-aeb7-e73f92feb172 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:28.894440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 57/112] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:24:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:24:28.895800 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475219b9-d55f-4d42-91f3-695fa88380d3 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:28.896095 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475219b9-d55f-4d42-91f3-695fa88380d3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:28.896378 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475219b9-d55f-4d42-91f3-695fa88380d3 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:28.897647 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 58/113] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:28.898802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbf744-2504-493d-9b3a-da519f20825f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:28.899092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbf744-2504-493d-9b3a-da519f20825f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:28.899373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbf744-2504-493d-9b3a-da519f20825f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:28.903090 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53eb85f-9f06-447a-aeb7-e73f92feb172 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:28.912902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-e53eb85f-9f06-447a-aeb7-e73f92feb172 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:24:28.912902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:28.912902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:28.912902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:28.912902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:28.912902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:28.912902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:28.912902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:28.912902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:28.912902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:28.912902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:28.912902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:28.912902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:28.912902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:28.912902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:28.912902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:28.913754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:28.913754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:28.913754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:28.913754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:28.913754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:28.913754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:28.913754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:28.913754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:28.913754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:28.913754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:28.913754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:28.913754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:24:28.913754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:28.913754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:28.913754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:28.913754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:28.914551 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:28.914551 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:28.914551 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:28.914551 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:28.914551 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:28.914551 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:28.914551 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:28.914551 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:28.914551 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:28.914551 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:28.914551 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:24:28.914551 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:24:28.916134 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 56/114] 10.222.0.67 () {56 vars in 1114 bytes} [Tue May 31 14:24:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:28.916299 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-82dbf744-2504-493d-9b3a-da519f20825f None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:28.917502 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-82dbf744-2504-493d-9b3a-da519f20825f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:28.919659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53eb85f-9f06-447a-aeb7-e73f92feb172 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:28.932509 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:28.989928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 59/115] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:28.991209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dbf744-2504-493d-9b3a-da519f20825f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:29.036392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53eb85f-9f06-447a-aeb7-e73f92feb172 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:29.073333 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d55794-841c-43e9-a1cc-f4bb0445c34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:29.074467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d55794-841c-43e9-a1cc-f4bb0445c34b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:29.075007 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d55794-841c-43e9-a1cc-f4bb0445c34b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:29.075514 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d55794-841c-43e9-a1cc-f4bb0445c34b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:29.087112 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d55794-841c-43e9-a1cc-f4bb0445c34b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:29.096248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-80d55794-841c-43e9-a1cc-f4bb0445c34b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 31 14:24:29.096248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:29.096248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:29.096248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:29.096248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:29.096248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:29.096248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:29.096248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:29.096248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:29.096248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:29.096248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:29.096248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:29.096248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:29.096248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:29.096248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:29.096248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:29.097081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:29.097081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:29.097081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:29.097081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:29.097081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:29.097081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:29.097081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:29.097081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:29.097081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:29.097081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:29.097081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:29.097081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:24:29.097081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:29.097081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:29.097081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:29.097081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:29.097928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:29.097928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:29.097928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:29.097928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:29.097928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:29.097928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:29.097928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:29.097928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:29.097928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:29.097928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:29.097928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 31 14:24:29.097928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:24:29.098512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dbf744-2504-493d-9b3a-da519f20825f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:29.099968 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 57/116] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:29.101336 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca262c-4a2b-44ae-998c-82c4943fa683 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:29.101633 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca262c-4a2b-44ae-998c-82c4943fa683 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:29.101916 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca262c-4a2b-44ae-998c-82c4943fa683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:29.118425 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-85ca262c-4a2b-44ae-998c-82c4943fa683 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:29.119682 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-85ca262c-4a2b-44ae-998c-82c4943fa683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:29.143605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e919b93-dbfd-499f-856f-08ff3032e540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:29.144489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e919b93-dbfd-499f-856f-08ff3032e540 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:29.144778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e919b93-dbfd-499f-856f-08ff3032e540 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:29.145062 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e919b93-dbfd-499f-856f-08ff3032e540 None admin] PATH_INFO: `/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:29.146311 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e919b93-dbfd-499f-856f-08ff3032e540 None admin] RBAC: Authorizing `identity:get_project(project_id=e3f26f512b354f6ebdbf6c8d3cc3e472)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:29.147804 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e919b93-dbfd-499f-856f-08ff3032e540 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:29.150793 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 60/117] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:28 2022] GET /identity/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:29.151593 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e919b93-dbfd-499f-856f-08ff3032e540 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:29.170157 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:29.175068 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:29.192269 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 58/118] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:29.193187 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca262c-4a2b-44ae-998c-82c4943fa683 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:29.274279 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e919b93-dbfd-499f-856f-08ff3032e540 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:29.308095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca262c-4a2b-44ae-998c-82c4943fa683 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:29.309426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76dfaf4-5165-40a5-a3a3-8ec5b3161515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:29.310101 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76dfaf4-5165-40a5-a3a3-8ec5b3161515 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:29.311584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76dfaf4-5165-40a5-a3a3-8ec5b3161515 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:29.311584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76dfaf4-5165-40a5-a3a3-8ec5b3161515 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:29.311961 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76dfaf4-5165-40a5-a3a3-8ec5b3161515 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:29.312269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76dfaf4-5165-40a5-a3a3-8ec5b3161515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:29.313639 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76dfaf4-5165-40a5-a3a3-8ec5b3161515 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:29.322577 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 61/119] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:24:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:29.323257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76dfaf4-5165-40a5-a3a3-8ec5b3161515 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:29.349461 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e8936-5eca-48ed-b993-c7ca51eda6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:29.350261 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e8936-5eca-48ed-b993-c7ca51eda6a1 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:29.350550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e8936-5eca-48ed-b993-c7ca51eda6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:29.350865 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e8936-5eca-48ed-b993-c7ca51eda6a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:29.361593 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e8936-5eca-48ed-b993-c7ca51eda6a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:29.369240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-a61e8936-5eca-48ed-b993-c7ca51eda6a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:24:29.369240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:29.369240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:29.369240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:29.369240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:29.369240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:29.369240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:29.369240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:29.369240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:29.369240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:29.369240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:29.369240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:29.369240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:29.369240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:29.369240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:29.369240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:29.370106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:29.370106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:29.370106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:29.370106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:29.370106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:29.370106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:29.370106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:29.370106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:29.370106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:29.370106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:29.370106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:29.370106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:24:29.370106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:29.370106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:29.370106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:29.370106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:29.377059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:29.377059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:29.377059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:29.377059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:29.377059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:29.377059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:29.377059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:29.377059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:29.377059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:29.377059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:29.377059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:24:29.377059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:24:29.377059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 59/120] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:24:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:29.377059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e8936-5eca-48ed-b993-c7ca51eda6a1 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:29.415526 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:29.423924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:29.440014 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76dfaf4-5165-40a5-a3a3-8ec5b3161515 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:29.477413 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a9493-d79d-4141-bcc8-7bb6331749bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:29.478116 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a9493-d79d-4141-bcc8-7bb6331749bb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:29.478410 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a9493-d79d-4141-bcc8-7bb6331749bb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:29.478721 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a9493-d79d-4141-bcc8-7bb6331749bb None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:29.479484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a9493-d79d-4141-bcc8-7bb6331749bb None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:29.479815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a9493-d79d-4141-bcc8-7bb6331749bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:29.481194 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a9493-d79d-4141-bcc8-7bb6331749bb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:29.489462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e8936-5eca-48ed-b993-c7ca51eda6a1 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:29.489940 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 62/121] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:24:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:29.490751 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a9493-d79d-4141-bcc8-7bb6331749bb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:29.530682 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd238a-c58a-4d07-8795-108107eee3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:29.531253 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd238a-c58a-4d07-8795-108107eee3ad None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:29.531544 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd238a-c58a-4d07-8795-108107eee3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:29.531827 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd238a-c58a-4d07-8795-108107eee3ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:29.532669 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd238a-c58a-4d07-8795-108107eee3ad None admin] RBAC: Adding query filter params (role.id=8655041bde1948a1be442c6239496c22, scope.project.id=e3f26f512b354f6ebdbf6c8d3cc3e472, user.id=2d38ae56e517482e86864dab992d1c22) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:29.532990 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd238a-c58a-4d07-8795-108107eee3ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:29.534564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd238a-c58a-4d07-8795-108107eee3ad None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:29.550691 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 60/122] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:24:29 2022] GET /identity/v3/role_assignments?user.id=2d38ae56e517482e86864dab992d1c22&scope.project.id=e3f26f512b354f6ebdbf6c8d3cc3e472&role.id=8655041bde1948a1be442c6239496c22 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:29.550691 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd238a-c58a-4d07-8795-108107eee3ad None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:29.622133 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a9493-d79d-4141-bcc8-7bb6331749bb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:29.661240 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd1483-ee0e-44aa-a886-bc6e0020114c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:29.662392 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd1483-ee0e-44aa-a886-bc6e0020114c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:29.663124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd1483-ee0e-44aa-a886-bc6e0020114c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:29.663484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd1483-ee0e-44aa-a886-bc6e0020114c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:29.674812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:29.680912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd1483-ee0e-44aa-a886-bc6e0020114c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:29.681643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd238a-c58a-4d07-8795-108107eee3ad None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:29.689738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:29.692478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-d9fd1483-ee0e-44aa-a886-bc6e0020114c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 31 14:24:29.692478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:29.692478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:29.692478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:29.692478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:29.692478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:29.692478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:29.692478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:29.692478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:29.692478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:29.692478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:29.692478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:29.692478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:29.692478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:29.692478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:29.692478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:29.693422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:29.693422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:29.693422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:29.693422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:29.693422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:29.693422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:29.693422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:29.693422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:29.693422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:29.693422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:29.693422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:29.693422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:24:29.693422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:29.693422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:29.693422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:29.693422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:29.694380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:29.694380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:29.694380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:29.694380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:29.694380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:29.694380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:29.694380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:29.694380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:29.694380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:29.694380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:29.694380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 31 14:24:29.694380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:24:29.697186 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 63/123] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:29.697186 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd1483-ee0e-44aa-a886-bc6e0020114c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:29.732808 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a51a99-b400-41f0-851e-de0c85b07897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:29.733742 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a51a99-b400-41f0-851e-de0c85b07897 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:29.734044 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a51a99-b400-41f0-851e-de0c85b07897 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:29.734327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a51a99-b400-41f0-851e-de0c85b07897 None admin] PATH_INFO: `/v3/users/f89c7583bd684f1a883ca3ccb2230e5b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:29.735793 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a51a99-b400-41f0-851e-de0c85b07897 None admin] RBAC: Authorizing `identity:get_user(user_id=f89c7583bd684f1a883ca3ccb2230e5b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:29.737981 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a51a99-b400-41f0-851e-de0c85b07897 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:29.741703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 61/124] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:29 2022] GET /identity/v3/users/f89c7583bd684f1a883ca3ccb2230e5b => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:29.742709 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a51a99-b400-41f0-851e-de0c85b07897 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:29.827881 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd1483-ee0e-44aa-a886-bc6e0020114c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:29.866237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eaaadf-d20e-479f-b602-84a76d1919f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:29.867132 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eaaadf-d20e-479f-b602-84a76d1919f4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:29.867528 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eaaadf-d20e-479f-b602-84a76d1919f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:29.867907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eaaadf-d20e-479f-b602-84a76d1919f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:29.868770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eaaadf-d20e-479f-b602-84a76d1919f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:29.869610 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eaaadf-d20e-479f-b602-84a76d1919f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:29.871398 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eaaadf-d20e-479f-b602-84a76d1919f4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:29.883707 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 64/125] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:29.884665 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eaaadf-d20e-479f-b602-84a76d1919f4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:29.890030 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a51a99-b400-41f0-851e-de0c85b07897 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:29.935650 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02598c4-4bfc-4f31-a0ba-321f276dfd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:29.936992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02598c4-4bfc-4f31-a0ba-321f276dfd13 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:29.937283 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02598c4-4bfc-4f31-a0ba-321f276dfd13 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:29.937567 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02598c4-4bfc-4f31-a0ba-321f276dfd13 None admin] PATH_INFO: `/v3/users/f89c7583bd684f1a883ca3ccb2230e5b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:29.939281 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02598c4-4bfc-4f31-a0ba-321f276dfd13 None admin] RBAC: Authorizing `identity:get_user(user_id=f89c7583bd684f1a883ca3ccb2230e5b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:29.941217 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02598c4-4bfc-4f31-a0ba-321f276dfd13 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:29.944159 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 62/126] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:29 2022] GET /identity/v3/users/f89c7583bd684f1a883ca3ccb2230e5b => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:29.945067 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02598c4-4bfc-4f31-a0ba-321f276dfd13 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:29.969946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:30.006265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:30.021070 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eaaadf-d20e-479f-b602-84a76d1919f4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:30.068591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fff02c-1c6e-41fc-97b3-db636501da06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:30.070184 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fff02c-1c6e-41fc-97b3-db636501da06 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:30.070716 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fff02c-1c6e-41fc-97b3-db636501da06 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:30.071181 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fff02c-1c6e-41fc-97b3-db636501da06 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:30.071944 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fff02c-1c6e-41fc-97b3-db636501da06 None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:30.072262 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fff02c-1c6e-41fc-97b3-db636501da06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:30.072450 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02598c4-4bfc-4f31-a0ba-321f276dfd13 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:30.073648 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fff02c-1c6e-41fc-97b3-db636501da06 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:30.086444 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 65/127] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:24:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:30.087841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fff02c-1c6e-41fc-97b3-db636501da06 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:30.113740 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd96a3c-b0ff-4cab-b5df-e7203d868714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:30.114637 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd96a3c-b0ff-4cab-b5df-e7203d868714 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:30.115051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd96a3c-b0ff-4cab-b5df-e7203d868714 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:30.115437 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd96a3c-b0ff-4cab-b5df-e7203d868714 None admin] PATH_INFO: `/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:30.116821 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd96a3c-b0ff-4cab-b5df-e7203d868714 None admin] RBAC: Authorizing `identity:get_project(project_id=e3f26f512b354f6ebdbf6c8d3cc3e472)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:30.119586 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd96a3c-b0ff-4cab-b5df-e7203d868714 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:30.125070 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 63/128] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:29 2022] GET /identity/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:30.125070 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd96a3c-b0ff-4cab-b5df-e7203d868714 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:30.234984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fff02c-1c6e-41fc-97b3-db636501da06 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:30.241921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd96a3c-b0ff-4cab-b5df-e7203d868714 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:30.253276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:30.262791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:30.275015 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf239fb-559e-4923-8b7f-a6718d4e7460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:30.276363 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf239fb-559e-4923-8b7f-a6718d4e7460 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:30.276970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf239fb-559e-4923-8b7f-a6718d4e7460 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:30.277453 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf239fb-559e-4923-8b7f-a6718d4e7460 None admin] PATH_INFO: `/v3/users/c01eab2db7454d578ce76aa7d022d460` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:30.279628 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf239fb-559e-4923-8b7f-a6718d4e7460 None admin] RBAC: Authorizing `identity:get_user(user_id=c01eab2db7454d578ce76aa7d022d460)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:30.281671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf239fb-559e-4923-8b7f-a6718d4e7460 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:30.282523 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de875463-51ee-4430-835e-e62b90de828c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:30.283469 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de875463-51ee-4430-835e-e62b90de828c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:30.283759 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de875463-51ee-4430-835e-e62b90de828c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:30.284043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de875463-51ee-4430-835e-e62b90de828c None admin] PATH_INFO: `/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:30.284657 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 66/129] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:30 2022] GET /identity/v3/users/c01eab2db7454d578ce76aa7d022d460 => generated 247 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:30.285775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf239fb-559e-4923-8b7f-a6718d4e7460 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:30.290385 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de875463-51ee-4430-835e-e62b90de828c None admin] RBAC: Authorizing `identity:get_project(project_id=e3f26f512b354f6ebdbf6c8d3cc3e472)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:30.291333 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de875463-51ee-4430-835e-e62b90de828c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:30.293220 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 64/130] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:30 2022] GET /identity/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:30.293820 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de875463-51ee-4430-835e-e62b90de828c None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:30.394622 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf239fb-559e-4923-8b7f-a6718d4e7460 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:30.408447 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de875463-51ee-4430-835e-e62b90de828c None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:30.431904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027d78d-6fbd-46fe-8a85-1f2be04479d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:30.433257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027d78d-6fbd-46fe-8a85-1f2be04479d1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:30.433809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027d78d-6fbd-46fe-8a85-1f2be04479d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:30.434285 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027d78d-6fbd-46fe-8a85-1f2be04479d1 None admin] PATH_INFO: `/v3/users/f89c7583bd684f1a883ca3ccb2230e5b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:30.435848 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f027d78d-6fbd-46fe-8a85-1f2be04479d1 None admin] RBAC: Authorizing `identity:get_user(user_id=f89c7583bd684f1a883ca3ccb2230e5b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:30.437628 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f027d78d-6fbd-46fe-8a85-1f2be04479d1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:30.439739 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 67/131] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:30 2022] GET /identity/v3/users/f89c7583bd684f1a883ca3ccb2230e5b => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:30.441004 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027d78d-6fbd-46fe-8a85-1f2be04479d1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:30.444690 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b56ea-da90-491d-8a25-9af44c1531cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:30.446180 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b56ea-da90-491d-8a25-9af44c1531cd None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:30.446180 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b56ea-da90-491d-8a25-9af44c1531cd None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:30.446369 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b56ea-da90-491d-8a25-9af44c1531cd None admin] PATH_INFO: `/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472/users/f89c7583bd684f1a883ca3ccb2230e5b/roles/1b9c18d4b8b540bf9825080748cec3bc` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:30.448588 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268b56ea-da90-491d-8a25-9af44c1531cd None admin] RBAC: Authorizing `identity:create_grant(project_id=e3f26f512b354f6ebdbf6c8d3cc3e472, user_id=f89c7583bd684f1a883ca3ccb2230e5b, role_id=1b9c18d4b8b540bf9825080748cec3bc)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:30.450044 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268b56ea-da90-491d-8a25-9af44c1531cd None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:30.470674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 65/132] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:24:30 2022] PUT /identity/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472/users/f89c7583bd684f1a883ca3ccb2230e5b/roles/1b9c18d4b8b540bf9825080748cec3bc => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:24:30.470674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b56ea-da90-491d-8a25-9af44c1531cd None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:30.555160 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:30.572180 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:30.640219 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b56ea-da90-491d-8a25-9af44c1531cd None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:30.687413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57309db-0809-4387-b512-fca446d86863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:30.688341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57309db-0809-4387-b512-fca446d86863 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:30.688641 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57309db-0809-4387-b512-fca446d86863 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:30.688930 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57309db-0809-4387-b512-fca446d86863 None admin] PATH_INFO: `/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472/users/f89c7583bd684f1a883ca3ccb2230e5b/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:30.691202 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57309db-0809-4387-b512-fca446d86863 None admin] RBAC: Authorizing `identity:create_grant(project_id=e3f26f512b354f6ebdbf6c8d3cc3e472, user_id=f89c7583bd684f1a883ca3ccb2230e5b, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:30.693177 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57309db-0809-4387-b512-fca446d86863 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:30.703819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027d78d-6fbd-46fe-8a85-1f2be04479d1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:30.715912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 66/133] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:24:30 2022] PUT /identity/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472/users/f89c7583bd684f1a883ca3ccb2230e5b/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:24:30.717265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57309db-0809-4387-b512-fca446d86863 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:30.748576 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd2c54-109b-449d-8209-622385b7ec06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:30.749841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd2c54-109b-449d-8209-622385b7ec06 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:30.750334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd2c54-109b-449d-8209-622385b7ec06 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:30.750871 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd2c54-109b-449d-8209-622385b7ec06 None admin] PATH_INFO: `/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:30.752745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd2c54-109b-449d-8209-622385b7ec06 None admin] RBAC: Authorizing `identity:get_project(project_id=e3f26f512b354f6ebdbf6c8d3cc3e472)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:30.754518 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd2c54-109b-449d-8209-622385b7ec06 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:30.757212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 68/134] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:30 2022] GET /identity/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472 => generated 298 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:30.774657 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd2c54-109b-449d-8209-622385b7ec06 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:30.844849 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:30.848861 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57309db-0809-4387-b512-fca446d86863 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:30.895759 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c329687-7145-4182-8750-6a4436dd7136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:30.902685 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c329687-7145-4182-8750-6a4436dd7136 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:30.902685 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c329687-7145-4182-8750-6a4436dd7136 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:30.902685 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c329687-7145-4182-8750-6a4436dd7136 None admin] PATH_INFO: `/v3/projects/1aac711712614ffd98b5667d5dc8bc91` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:30.909417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c329687-7145-4182-8750-6a4436dd7136 None admin] RBAC: Authorizing `identity:get_project(project_id=1aac711712614ffd98b5667d5dc8bc91)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:30.912051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c329687-7145-4182-8750-6a4436dd7136 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:30.915560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 67/135] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:30 2022] GET /identity/v3/projects/1aac711712614ffd98b5667d5dc8bc91 => generated 312 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:30.920335 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8ec28-aec6-4d07-a222-a0bf0cf3cc84 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:30.920627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8ec28-aec6-4d07-a222-a0bf0cf3cc84 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:30.920912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8ec28-aec6-4d07-a222-a0bf0cf3cc84 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:30.922478 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 68/136] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:30.923808 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8ec28-aec6-4d07-a222-a0bf0cf3cc84 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:30.949277 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd2c54-109b-449d-8209-622385b7ec06 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:30.980398 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:31.000765 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178b3ac-3a38-41bd-8538-d287541736f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:31.002166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178b3ac-3a38-41bd-8538-d287541736f7 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:31.002679 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178b3ac-3a38-41bd-8538-d287541736f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:31.003163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178b3ac-3a38-41bd-8538-d287541736f7 None admin] PATH_INFO: `/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472/users/c01eab2db7454d578ce76aa7d022d460/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:31.012397 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178b3ac-3a38-41bd-8538-d287541736f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=e3f26f512b354f6ebdbf6c8d3cc3e472, user_id=c01eab2db7454d578ce76aa7d022d460, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:31.014942 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178b3ac-3a38-41bd-8538-d287541736f7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:31.047267 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 69/137] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:24:30 2022] PUT /identity/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472/users/c01eab2db7454d578ce76aa7d022d460/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:24:31.048933 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dd6a5-0e4d-4ced-a197-eca70e2e0d1d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:31.049432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dd6a5-0e4d-4ced-a197-eca70e2e0d1d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:31.050027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dd6a5-0e4d-4ced-a197-eca70e2e0d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:31.055382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8ec28-aec6-4d07-a222-a0bf0cf3cc84 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:31.083631 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8a3dd6a5-0e4d-4ced-a197-eca70e2e0d1d None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:31.086390 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8a3dd6a5-0e4d-4ced-a197-eca70e2e0d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:31.141097 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e1242-a2ac-4b5b-9c5e-c026e9eacceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:31.142077 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e1242-a2ac-4b5b-9c5e-c026e9eacceb None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:31.142372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e1242-a2ac-4b5b-9c5e-c026e9eacceb None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:31.142673 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e1242-a2ac-4b5b-9c5e-c026e9eacceb None admin] PATH_INFO: `/v3/projects/1aac711712614ffd98b5667d5dc8bc91/users/f89c7583bd684f1a883ca3ccb2230e5b/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:31.145820 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e1242-a2ac-4b5b-9c5e-c026e9eacceb None admin] RBAC: Authorizing `identity:create_grant(project_id=1aac711712614ffd98b5667d5dc8bc91, user_id=f89c7583bd684f1a883ca3ccb2230e5b, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:31.148926 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e1242-a2ac-4b5b-9c5e-c026e9eacceb None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:31.179255 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 69/138] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:24:30 2022] PUT /identity/v3/projects/1aac711712614ffd98b5667d5dc8bc91/users/f89c7583bd684f1a883ca3ccb2230e5b/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:24:31.211945 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 70/139] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:31.238748 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e1242-a2ac-4b5b-9c5e-c026e9eacceb None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:31.275547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:31.368775 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e1242-a2ac-4b5b-9c5e-c026e9eacceb None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:31.408461 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869fa6f3-c442-4e89-93e8-fcb683571b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:31.409200 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fa6f3-c442-4e89-93e8-fcb683571b78 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:31.409545 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fa6f3-c442-4e89-93e8-fcb683571b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:31.409824 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fa6f3-c442-4e89-93e8-fcb683571b78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:31.411098 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869fa6f3-c442-4e89-93e8-fcb683571b78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:31.412635 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869fa6f3-c442-4e89-93e8-fcb683571b78 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:31.414692 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 70/140] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:24:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:31.420361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3dd6a5-0e4d-4ced-a197-eca70e2e0d1d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:31.553211 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3dd6a5-0e4d-4ced-a197-eca70e2e0d1d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:31.572883 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:31.593004 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb0d33-90a5-4734-b301-3a963690d6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:31.593700 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb0d33-90a5-4734-b301-3a963690d6b9 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:31.593993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb0d33-90a5-4734-b301-3a963690d6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:31.594274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb0d33-90a5-4734-b301-3a963690d6b9 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:31.595042 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb0d33-90a5-4734-b301-3a963690d6b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:31.596512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb0d33-90a5-4734-b301-3a963690d6b9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:31.622249 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 71/141] 10.222.0.67 () {60 vars in 1132 bytes} [Tue May 31 14:24:31 2022] POST /identity/v3/users => generated 283 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:24:32.109242 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58275a-c28f-4211-9921-40506c6b9ac1 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:32.109736 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58275a-c28f-4211-9921-40506c6b9ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:32.109833 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58275a-c28f-4211-9921-40506c6b9ac1 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:32.111213 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 71/142] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:32.116481 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba3e04-121d-4f5f-803c-558c4d18bcbb None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:32.116787 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba3e04-121d-4f5f-803c-558c4d18bcbb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:32.117079 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba3e04-121d-4f5f-803c-558c4d18bcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:32.137320 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ebba3e04-121d-4f5f-803c-558c4d18bcbb None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:32.139134 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ebba3e04-121d-4f5f-803c-558c4d18bcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:32.225625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 72/143] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:32.248682 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58275a-c28f-4211-9921-40506c6b9ac1 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:32.355206 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:32.364933 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58275a-c28f-4211-9921-40506c6b9ac1 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:32.401488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1639afe-2509-49c6-9d53-0e3b263bbaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:32.402301 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1639afe-2509-49c6-9d53-0e3b263bbaff None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:32.402590 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1639afe-2509-49c6-9d53-0e3b263bbaff None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:32.402905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1639afe-2509-49c6-9d53-0e3b263bbaff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:32.416336 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1639afe-2509-49c6-9d53-0e3b263bbaff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:32.424902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-a1639afe-2509-49c6-9d53-0e3b263bbaff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 31 14:24:32.424902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:32.424902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:32.424902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:32.424902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:32.424902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:32.424902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:32.424902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:32.424902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:32.424902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:32.424902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:32.424902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:32.424902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:32.424902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:32.424902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:32.424902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:32.425823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:32.425823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:32.425823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:32.425823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:32.425823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:32.425823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:32.425823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:32.425823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:32.425823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:32.425823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:32.425823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:32.425823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:24:32.425823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:32.425823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:32.425823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:32.425823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:32.427009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:32.427009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:32.427009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:32.427009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:32.427009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:32.427009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:32.427009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:32.427009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:32.427009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:32.427009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:32.427009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 31 14:24:32.427009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:24:32.427009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 72/144] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:32.431958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba3e04-121d-4f5f-803c-558c4d18bcbb None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:32.552843 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba3e04-121d-4f5f-803c-558c4d18bcbb None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:32.555427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:32.594970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a2f69-52ca-4ca4-8ab9-bf19324667cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:32.595726 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a2f69-52ca-4ca4-8ab9-bf19324667cd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:32.595957 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a2f69-52ca-4ca4-8ab9-bf19324667cd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:32.596244 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a2f69-52ca-4ca4-8ab9-bf19324667cd None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:32.597015 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a2f69-52ca-4ca4-8ab9-bf19324667cd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:32.597332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a2f69-52ca-4ca4-8ab9-bf19324667cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:32.598781 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a2f69-52ca-4ca4-8ab9-bf19324667cd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:32.609144 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 73/145] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:24:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:32.614693 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1639afe-2509-49c6-9d53-0e3b263bbaff None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:32.737682 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1639afe-2509-49c6-9d53-0e3b263bbaff None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:32.776844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b78399-ab16-4326-b9a2-be1ccc963b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:32.777743 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b78399-ab16-4326-b9a2-be1ccc963b3a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:32.778031 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b78399-ab16-4326-b9a2-be1ccc963b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:32.778316 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b78399-ab16-4326-b9a2-be1ccc963b3a None admin] PATH_INFO: `/v3/users/2d38ae56e517482e86864dab992d1c22` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:32.783396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b78399-ab16-4326-b9a2-be1ccc963b3a None admin] RBAC: Authorizing `identity:get_user(user_id=2d38ae56e517482e86864dab992d1c22)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:32.784882 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b78399-ab16-4326-b9a2-be1ccc963b3a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:32.786562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 73/146] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:32 2022] GET /identity/v3/users/2d38ae56e517482e86864dab992d1c22 => generated 289 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:32.792114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a2f69-52ca-4ca4-8ab9-bf19324667cd None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:32.919720 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a2f69-52ca-4ca4-8ab9-bf19324667cd None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:32.959851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb090996-be86-4235-8d42-7fc4df2dd68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:32.960743 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb090996-be86-4235-8d42-7fc4df2dd68a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:32.961036 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb090996-be86-4235-8d42-7fc4df2dd68a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:32.961325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb090996-be86-4235-8d42-7fc4df2dd68a None admin] PATH_INFO: `/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:32.967407 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb090996-be86-4235-8d42-7fc4df2dd68a None admin] RBAC: Authorizing `identity:get_project(project_id=e3f26f512b354f6ebdbf6c8d3cc3e472)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:32.968900 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb090996-be86-4235-8d42-7fc4df2dd68a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:32.970673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 74/147] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:32 2022] GET /identity/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:32.976511 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b78399-ab16-4326-b9a2-be1ccc963b3a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:33.003174 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:33.085336 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b78399-ab16-4326-b9a2-be1ccc963b3a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:33.128187 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384eb4b-14f6-47b5-9509-df35983dfc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:33.129570 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384eb4b-14f6-47b5-9509-df35983dfc98 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:33.129570 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384eb4b-14f6-47b5-9509-df35983dfc98 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:33.129728 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384eb4b-14f6-47b5-9509-df35983dfc98 None admin] PATH_INFO: `/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472/users/2d38ae56e517482e86864dab992d1c22/roles/8655041bde1948a1be442c6239496c22` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:33.135530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384eb4b-14f6-47b5-9509-df35983dfc98 None admin] RBAC: Authorizing `identity:create_grant(project_id=e3f26f512b354f6ebdbf6c8d3cc3e472, user_id=2d38ae56e517482e86864dab992d1c22, role_id=8655041bde1948a1be442c6239496c22)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:33.137020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384eb4b-14f6-47b5-9509-df35983dfc98 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:33.163495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 74/148] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:24:32 2022] PUT /identity/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472/users/2d38ae56e517482e86864dab992d1c22/roles/8655041bde1948a1be442c6239496c22 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:24:34.506193 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54997f6-28b6-416a-b710-eff6ca26ebca None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:34.507208 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54997f6-28b6-416a-b710-eff6ca26ebca None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:34.507781 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54997f6-28b6-416a-b710-eff6ca26ebca None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:34.509387 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 75/149] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:34.524681 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3769a-e711-4f82-98ec-bada1feef3cf None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:34.525622 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3769a-e711-4f82-98ec-bada1feef3cf None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:34.526123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3769a-e711-4f82-98ec-bada1feef3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:34.545367 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-f6b3769a-e711-4f82-98ec-bada1feef3cf None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:34.546978 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f6b3769a-e711-4f82-98ec-bada1feef3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:34.618383 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 75/150] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:34.646334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54997f6-28b6-416a-b710-eff6ca26ebca None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:34.690168 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:34.771709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54997f6-28b6-416a-b710-eff6ca26ebca None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:34.818067 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8575404-e2d9-4faf-a5fe-0a6999076c17 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:34.818727 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8575404-e2d9-4faf-a5fe-0a6999076c17 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:34.819209 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8575404-e2d9-4faf-a5fe-0a6999076c17 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:34.820790 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 76/151] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 31 14:24:34.821289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d9e2d-4e23-4443-bd70-328085d6562c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:34.822350 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d9e2d-4e23-4443-bd70-328085d6562c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:34.822855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d9e2d-4e23-4443-bd70-328085d6562c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:34.823332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d9e2d-4e23-4443-bd70-328085d6562c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:34.826467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecfd6a-5cf4-43b1-8ca1-7ab1a556228d None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:34.826983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecfd6a-5cf4-43b1-8ca1-7ab1a556228d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:34.827738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecfd6a-5cf4-43b1-8ca1-7ab1a556228d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:34.851285 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-e0ecfd6a-5cf4-43b1-8ca1-7ab1a556228d None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:34.855492 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e0ecfd6a-5cf4-43b1-8ca1-7ab1a556228d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:34.858541 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d9e2d-4e23-4443-bd70-328085d6562c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:34.866617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-9e1d9e2d-4e23-4443-bd70-328085d6562c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 31 14:24:34.866617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:34.866617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:34.866617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:34.866617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:34.866617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:34.866617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:34.866617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:34.866617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:34.866617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:34.866617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:34.866617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:34.866617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:34.866617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:34.866617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:34.866617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:34.867489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:34.867489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:34.867489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:34.867489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:34.867489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:34.867489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:34.867489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:34.867489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:34.867489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:34.867489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:34.867489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:34.867489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:24:34.867489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:34.867489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:34.867489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:34.867489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:34.868254 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:34.868254 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:34.868254 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:34.868254 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:34.868254 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:34.868254 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:34.868254 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:34.868254 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:34.868254 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:34.868254 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:34.868254 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 31 14:24:34.868254 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:24:34.877051 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 76/152] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:34.903597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:34.905891 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d9e2d-4e23-4443-bd70-328085d6562c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:34.991923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 77/153] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 31 14:24:35.043224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:35.044586 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ecfd6a-5cf4-43b1-8ca1-7ab1a556228d None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:35.066587 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d9e2d-4e23-4443-bd70-328085d6562c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:35.167889 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ecfd6a-5cf4-43b1-8ca1-7ab1a556228d None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:35.172572 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff93566-4636-4f48-89d2-ec1b168056a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:35.173438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff93566-4636-4f48-89d2-ec1b168056a0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:35.173924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff93566-4636-4f48-89d2-ec1b168056a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:35.174344 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff93566-4636-4f48-89d2-ec1b168056a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:35.175268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff93566-4636-4f48-89d2-ec1b168056a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:35.175720 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff93566-4636-4f48-89d2-ec1b168056a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:35.177260 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff93566-4636-4f48-89d2-ec1b168056a0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:35.196864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 77/154] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:24:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:35.212083 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff93566-4636-4f48-89d2-ec1b168056a0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:35.219412 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a6386-079b-49c9-b568-ed57b25334e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:35.220245 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a6386-079b-49c9-b568-ed57b25334e3 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:35.220528 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a6386-079b-49c9-b568-ed57b25334e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:35.220805 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a6386-079b-49c9-b568-ed57b25334e3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:35.243796 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06a6386-079b-49c9-b568-ed57b25334e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:35.252558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-d06a6386-079b-49c9-b568-ed57b25334e3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:24:35.252558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:35.252558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:35.252558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:35.252558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:35.252558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:35.252558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:35.252558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:35.252558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:35.252558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:35.252558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:35.252558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:35.252558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:35.252558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:35.252558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:35.252558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:35.253393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:35.253393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:35.253393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:35.253393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:35.253393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:35.253393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:35.253393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:35.253393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:35.253393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:35.253393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:35.253393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:35.253393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:24:35.253393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:35.253393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:35.253393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:35.253393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:35.254341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:35.254341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:35.254341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:35.254341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:35.254341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:35.254341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:35.254341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:35.254341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:35.254341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:35.254341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:35.254341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:24:35.254341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:24:35.254341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 78/155] 10.222.0.67 () {56 vars in 1114 bytes} [Tue May 31 14:24:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:35.256720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552244d9-01fe-48e0-8cf5-9a664b9ad187 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:35.259284 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552244d9-01fe-48e0-8cf5-9a664b9ad187 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:35.262312 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552244d9-01fe-48e0-8cf5-9a664b9ad187 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:35.265337 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 79/156] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:35 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:35.266101 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552244d9-01fe-48e0-8cf5-9a664b9ad187 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:35.274965 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:35.373536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff93566-4636-4f48-89d2-ec1b168056a0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:35.416888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d303d8-c911-4e2a-99c6-9a94a4f2d677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:35.417783 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d303d8-c911-4e2a-99c6-9a94a4f2d677 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:35.418070 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d303d8-c911-4e2a-99c6-9a94a4f2d677 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:35.418348 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d303d8-c911-4e2a-99c6-9a94a4f2d677 None admin] PATH_INFO: `/v3/users/f89c7583bd684f1a883ca3ccb2230e5b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:35.422303 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d303d8-c911-4e2a-99c6-9a94a4f2d677 None admin] RBAC: Authorizing `identity:get_user(user_id=f89c7583bd684f1a883ca3ccb2230e5b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:35.424674 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d303d8-c911-4e2a-99c6-9a94a4f2d677 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:35.430678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 78/157] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:35 2022] GET /identity/v3/users/f89c7583bd684f1a883ca3ccb2230e5b => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:35.433173 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c871e-5472-4a12-8bd9-b78057a02178 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:35.433724 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c871e-5472-4a12-8bd9-b78057a02178 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:35.434154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c871e-5472-4a12-8bd9-b78057a02178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:35.464427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-385c871e-5472-4a12-8bd9-b78057a02178 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:35.465789 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-385c871e-5472-4a12-8bd9-b78057a02178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:35.485713 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552244d9-01fe-48e0-8cf5-9a664b9ad187 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:35.540543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:35.544847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9b77e-9919-43f5-9ef7-ee9ffa3dadff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:35.546196 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9b77e-9919-43f5-9ef7-ee9ffa3dadff None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:35.546699 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9b77e-9919-43f5-9ef7-ee9ffa3dadff None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:35.547201 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9b77e-9919-43f5-9ef7-ee9ffa3dadff None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:35.548492 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9b77e-9919-43f5-9ef7-ee9ffa3dadff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:35.549008 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9b77e-9919-43f5-9ef7-ee9ffa3dadff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:35.550628 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9b77e-9919-43f5-9ef7-ee9ffa3dadff None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:35.560669 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 80/158] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:24:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:35.561932 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9b77e-9919-43f5-9ef7-ee9ffa3dadff None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:35.672659 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 79/159] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:35.673499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c871e-5472-4a12-8bd9-b78057a02178 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:35.706674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:35.708954 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9b77e-9919-43f5-9ef7-ee9ffa3dadff None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:35.752086 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49df4bd-0f1a-4f03-be00-2678bc006858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:35.753135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49df4bd-0f1a-4f03-be00-2678bc006858 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:35.753619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49df4bd-0f1a-4f03-be00-2678bc006858 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:35.754109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49df4bd-0f1a-4f03-be00-2678bc006858 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:35.861872 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c871e-5472-4a12-8bd9-b78057a02178 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:35.907725 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49df4bd-0f1a-4f03-be00-2678bc006858 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:35.909222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49df4bd-0f1a-4f03-be00-2678bc006858 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:35.929204 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35ceac-3b76-40d5-b45a-cd373bf113ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:35.930792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35ceac-3b76-40d5-b45a-cd373bf113ec None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:35.931539 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35ceac-3b76-40d5-b45a-cd373bf113ec None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:35.932040 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35ceac-3b76-40d5-b45a-cd373bf113ec None admin] PATH_INFO: `/v3/users/f89c7583bd684f1a883ca3ccb2230e5b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:35.933612 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35ceac-3b76-40d5-b45a-cd373bf113ec None admin] RBAC: Authorizing `identity:get_user(user_id=f89c7583bd684f1a883ca3ccb2230e5b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:35.935933 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35ceac-3b76-40d5-b45a-cd373bf113ec None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:35.938449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 80/160] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:35 2022] GET /identity/v3/users/f89c7583bd684f1a883ca3ccb2230e5b => generated 275 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:35.946666 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35ceac-3b76-40d5-b45a-cd373bf113ec None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:35.990497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:36.015054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:36.062931 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 81/161] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:24:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:24:36.063807 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49df4bd-0f1a-4f03-be00-2678bc006858 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:36.141375 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35ceac-3b76-40d5-b45a-cd373bf113ec None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:36.209984 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49df4bd-0f1a-4f03-be00-2678bc006858 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:36.215329 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac744d5-a52a-422d-925d-153f56630a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:36.216151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac744d5-a52a-422d-925d-153f56630a5d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:36.216973 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac744d5-a52a-422d-925d-153f56630a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:36.217249 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac744d5-a52a-422d-925d-153f56630a5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:36.243525 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac744d5-a52a-422d-925d-153f56630a5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:36.259633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-7ac744d5-a52a-422d-925d-153f56630a5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:24:36.259633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:36.259633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:36.259633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:36.259633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:36.259633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:36.259633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:36.259633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:36.259633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:36.259633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:36.259633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:36.259633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:36.259633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:36.259633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:36.259633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:36.259633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:36.260600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:36.260600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:36.260600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:36.260600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:36.260600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:36.260600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:36.260600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:36.260600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:36.260600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:36.260600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:36.260600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:36.260600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:24:36.260600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:36.260600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:36.260600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:36.260600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:36.261379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:36.261379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:36.261379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:36.261379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:36.261379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:36.261379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:36.261379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:36.261379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:36.261379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:36.261379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:36.261379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:24:36.261379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:24:36.262446 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 81/162] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:24:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:36.263671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac744d5-a52a-422d-925d-153f56630a5d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:36.266977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b82271-f041-46d8-8b27-ca1261c07f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:36.267654 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b82271-f041-46d8-8b27-ca1261c07f9e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:36.267941 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b82271-f041-46d8-8b27-ca1261c07f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:36.268220 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b82271-f041-46d8-8b27-ca1261c07f9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:36.339036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:36.368996 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b82271-f041-46d8-8b27-ca1261c07f9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:36.370588 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b82271-f041-46d8-8b27-ca1261c07f9e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:36.417593 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:36.464043 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac744d5-a52a-422d-925d-153f56630a5d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:36.531796 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bc3d1-1cf2-42bc-a876-6d8152185f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:36.532683 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bc3d1-1cf2-42bc-a876-6d8152185f97 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:36.532965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bc3d1-1cf2-42bc-a876-6d8152185f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:36.533238 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bc3d1-1cf2-42bc-a876-6d8152185f97 None admin] PATH_INFO: `/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:36.535608 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286bc3d1-1cf2-42bc-a876-6d8152185f97 None admin] RBAC: Authorizing `identity:get_project(project_id=e3f26f512b354f6ebdbf6c8d3cc3e472)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:36.537063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286bc3d1-1cf2-42bc-a876-6d8152185f97 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:36.540099 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 82/163] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:36 2022] GET /identity/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472 => generated 298 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:36.540891 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bc3d1-1cf2-42bc-a876-6d8152185f97 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:36.580390 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 82/164] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:24:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:24:36.581820 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeed718-3dce-44c9-af60-39bb84553d54 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:36.582112 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeed718-3dce-44c9-af60-39bb84553d54 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:36.582390 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeed718-3dce-44c9-af60-39bb84553d54 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:36.583836 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 83/165] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:36.584815 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdbb07-145e-44e1-b10e-13502197c7b6 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:36.585098 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdbb07-145e-44e1-b10e-13502197c7b6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:36.585370 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdbb07-145e-44e1-b10e-13502197c7b6 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:36.590609 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 84/166] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:36.591950 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bdbb07-145e-44e1-b10e-13502197c7b6 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:36.667034 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bc3d1-1cf2-42bc-a876-6d8152185f97 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:36.703022 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:36.707750 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3845f45-8926-4e38-8123-e944e9f05822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:36.708601 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3845f45-8926-4e38-8123-e944e9f05822 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:36.708712 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3845f45-8926-4e38-8123-e944e9f05822 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:36.708985 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3845f45-8926-4e38-8123-e944e9f05822 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:36.709756 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3845f45-8926-4e38-8123-e944e9f05822 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:36.710069 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3845f45-8926-4e38-8123-e944e9f05822 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:36.711581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3845f45-8926-4e38-8123-e944e9f05822 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:36.729239 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 83/167] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:36.729852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3845f45-8926-4e38-8123-e944e9f05822 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:36.760966 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:36.782458 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bdbb07-145e-44e1-b10e-13502197c7b6 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:36.826379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979846b7-12a2-4589-809c-99eb01857dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:36.827788 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979846b7-12a2-4589-809c-99eb01857dcc None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:36.828425 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979846b7-12a2-4589-809c-99eb01857dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:36.828858 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979846b7-12a2-4589-809c-99eb01857dcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:36.830681 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979846b7-12a2-4589-809c-99eb01857dcc None admin] RBAC: Adding query filter params (role.id=9ce8fdecf52d482d894fdeec4f530624, scope.project.id=e3f26f512b354f6ebdbf6c8d3cc3e472, user.id=f89c7583bd684f1a883ca3ccb2230e5b) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:36.831173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979846b7-12a2-4589-809c-99eb01857dcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:36.833633 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979846b7-12a2-4589-809c-99eb01857dcc None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:36.843736 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 85/168] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:24:36 2022] GET /identity/v3/role_assignments?user.id=f89c7583bd684f1a883ca3ccb2230e5b&scope.project.id=e3f26f512b354f6ebdbf6c8d3cc3e472&role.id=9ce8fdecf52d482d894fdeec4f530624 => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:36.846812 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ed284-79c3-448a-b215-a1b75a3ce6c0 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:36.847255 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ed284-79c3-448a-b215-a1b75a3ce6c0 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:36.847696 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ed284-79c3-448a-b215-a1b75a3ce6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:36.873364 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6b6ed284-79c3-448a-b215-a1b75a3ce6c0 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:36.874617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6b6ed284-79c3-448a-b215-a1b75a3ce6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:36.957270 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 86/169] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:36.958739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb3ef4-4a9a-41b9-a68c-7712084b48a5 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:36.959043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb3ef4-4a9a-41b9-a68c-7712084b48a5 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:36.959913 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb3ef4-4a9a-41b9-a68c-7712084b48a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:36.977851 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-9adb3ef4-4a9a-41b9-a68c-7712084b48a5 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:36.979099 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9adb3ef4-4a9a-41b9-a68c-7712084b48a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:37.055183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:37.060744 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 87/170] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:37.067485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb3ef4-4a9a-41b9-a68c-7712084b48a5 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:37.088737 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3845f45-8926-4e38-8123-e944e9f05822 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:37.175322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df292be-6dd6-43d9-9aca-f39342583316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:37.176562 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df292be-6dd6-43d9-9aca-f39342583316 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:37.177042 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df292be-6dd6-43d9-9aca-f39342583316 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:37.177507 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df292be-6dd6-43d9-9aca-f39342583316 None admin] PATH_INFO: `/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:37.179084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df292be-6dd6-43d9-9aca-f39342583316 None admin] RBAC: Authorizing `identity:get_project(project_id=e3f26f512b354f6ebdbf6c8d3cc3e472)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:37.180843 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df292be-6dd6-43d9-9aca-f39342583316 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:37.184075 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 84/171] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:36 2022] GET /identity/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472 => generated 298 bytes in 455 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:37.186523 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df292be-6dd6-43d9-9aca-f39342583316 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:37.261950 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb3ef4-4a9a-41b9-a68c-7712084b48a5 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:37.312442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:37.319780 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b167f-ae22-4a80-8860-ca22a63b9ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:37.321383 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b167f-ae22-4a80-8860-ca22a63b9ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:37.321859 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b167f-ae22-4a80-8860-ca22a63b9ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:37.322572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b167f-ae22-4a80-8860-ca22a63b9ad8 None admin] PATH_INFO: `/v3/users/c01eab2db7454d578ce76aa7d022d460` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:37.324507 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b167f-ae22-4a80-8860-ca22a63b9ad8 None admin] RBAC: Authorizing `identity:get_user(user_id=c01eab2db7454d578ce76aa7d022d460)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:37.327142 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b167f-ae22-4a80-8860-ca22a63b9ad8 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:37.329983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 88/172] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:37 2022] GET /identity/v3/users/c01eab2db7454d578ce76aa7d022d460 => generated 247 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:37.335716 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b167f-ae22-4a80-8860-ca22a63b9ad8 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:37.338097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df292be-6dd6-43d9-9aca-f39342583316 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:37.373384 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b47309-1d98-4e30-90a9-95153700d72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:37.374282 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b47309-1d98-4e30-90a9-95153700d72a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:37.374634 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b47309-1d98-4e30-90a9-95153700d72a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:37.374947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b47309-1d98-4e30-90a9-95153700d72a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:37.384912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b47309-1d98-4e30-90a9-95153700d72a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:37.392670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-62b47309-1d98-4e30-90a9-95153700d72a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 31 14:24:37.392670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:37.392670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:37.392670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:37.392670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:37.392670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:37.392670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:37.392670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:37.392670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:37.392670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:37.392670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:37.392670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:37.392670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:37.392670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:37.392670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:37.392670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:37.393543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:37.393543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:37.393543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:37.393543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:37.393543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:37.393543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:37.393543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:37.393543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:37.393543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:37.393543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:37.393543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:37.393543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:24:37.393543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:37.393543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:37.393543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:37.393543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:37.394426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:37.394426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:37.394426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:37.394426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:37.394426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:37.394426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:37.394426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:37.394426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:37.394426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:37.394426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:37.394426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 31 14:24:37.394426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:24:37.394426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 85/173] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:37.395119 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b47309-1d98-4e30-90a9-95153700d72a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:37.398947 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:37.443570 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b167f-ae22-4a80-8860-ca22a63b9ad8 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:37.476646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67733982-7dac-42b4-b7cc-6d0d92b3151d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:37.477437 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67733982-7dac-42b4-b7cc-6d0d92b3151d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:37.477721 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67733982-7dac-42b4-b7cc-6d0d92b3151d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:37.478000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67733982-7dac-42b4-b7cc-6d0d92b3151d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:37.491530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67733982-7dac-42b4-b7cc-6d0d92b3151d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:37.501962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-67733982-7dac-42b4-b7cc-6d0d92b3151d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:24:37.501962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:37.501962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:37.501962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:37.501962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:37.501962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:37.501962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:37.501962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:37.501962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:37.501962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:37.501962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:37.501962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:37.501962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:37.501962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:37.501962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:37.501962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:37.502892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:37.502892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:37.502892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:37.502892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:37.502892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:37.502892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:37.502892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:37.502892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:37.502892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:37.502892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:37.502892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:37.502892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:24:37.502892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:37.502892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:37.502892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:37.502892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:37.503886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:37.503886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:37.503886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:37.503886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:37.503886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:37.503886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:37.503886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:37.503886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:37.503886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:37.503886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:37.503886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:24:37.503886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:24:37.504675 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b47309-1d98-4e30-90a9-95153700d72a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:37.504756 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 89/174] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:24:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:37.504756 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3a267-be76-46ab-9da1-a9e5c8624091 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:37.504756 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3a267-be76-46ab-9da1-a9e5c8624091 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:37.505056 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3a267-be76-46ab-9da1-a9e5c8624091 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:37.506302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 90/175] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:37.510387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc620f-f4c9-47cc-9eb4-7fa96f94a96e None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:37.510703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc620f-f4c9-47cc-9eb4-7fa96f94a96e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:37.510981 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc620f-f4c9-47cc-9eb4-7fa96f94a96e None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:37.512273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 91/176] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:37.512968 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc620f-f4c9-47cc-9eb4-7fa96f94a96e None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:37.549136 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15546a9-9859-48b6-8d98-8b684a77635e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:37.549804 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15546a9-9859-48b6-8d98-8b684a77635e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:37.550100 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15546a9-9859-48b6-8d98-8b684a77635e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:37.550389 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15546a9-9859-48b6-8d98-8b684a77635e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:37.551274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15546a9-9859-48b6-8d98-8b684a77635e None admin] RBAC: Adding query filter params (role.id=1b9c18d4b8b540bf9825080748cec3bc, scope.project.id=e3f26f512b354f6ebdbf6c8d3cc3e472, user.id=f89c7583bd684f1a883ca3ccb2230e5b) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:37.551595 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15546a9-9859-48b6-8d98-8b684a77635e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:37.553058 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15546a9-9859-48b6-8d98-8b684a77635e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:37.559249 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 86/177] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:24:37 2022] GET /identity/v3/role_assignments?user.id=f89c7583bd684f1a883ca3ccb2230e5b&scope.project.id=e3f26f512b354f6ebdbf6c8d3cc3e472&role.id=1b9c18d4b8b540bf9825080748cec3bc => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:37.560503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a2fb0-88f8-4dc4-8f5e-8d32299922fa None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:37.560797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a2fb0-88f8-4dc4-8f5e-8d32299922fa None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:37.561087 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a2fb0-88f8-4dc4-8f5e-8d32299922fa None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:37.562545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 87/178] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:37.563054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a2fb0-88f8-4dc4-8f5e-8d32299922fa None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:37.572692 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:37.635678 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc620f-f4c9-47cc-9eb4-7fa96f94a96e None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:37.653448 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:37.677635 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523a2fc-fb58-45d2-8bd6-c546b50e2d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:37.678305 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523a2fc-fb58-45d2-8bd6-c546b50e2d98 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:37.678587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523a2fc-fb58-45d2-8bd6-c546b50e2d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:37.678890 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523a2fc-fb58-45d2-8bd6-c546b50e2d98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:37.687945 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a2fb0-88f8-4dc4-8f5e-8d32299922fa None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:37.725246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571776f-e108-41fb-a5d6-6b5fc908545f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:37.725933 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571776f-e108-41fb-a5d6-6b5fc908545f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:37.726391 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571776f-e108-41fb-a5d6-6b5fc908545f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:37.726496 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571776f-e108-41fb-a5d6-6b5fc908545f None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:37.727288 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f571776f-e108-41fb-a5d6-6b5fc908545f None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:37.727600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f571776f-e108-41fb-a5d6-6b5fc908545f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:37.729033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f571776f-e108-41fb-a5d6-6b5fc908545f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:37.741065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 88/179] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:24:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:37.741764 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571776f-e108-41fb-a5d6-6b5fc908545f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:37.758005 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523a2fc-fb58-45d2-8bd6-c546b50e2d98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:37.759527 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523a2fc-fb58-45d2-8bd6-c546b50e2d98 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:37.826533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:37.868262 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571776f-e108-41fb-a5d6-6b5fc908545f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:37.881554 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 92/180] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:24:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:24:37.882885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d179a-5893-4ab4-ab4d-f5e02d8cc477 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:37.883501 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d179a-5893-4ab4-ab4d-f5e02d8cc477 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:37.883730 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d179a-5893-4ab4-ab4d-f5e02d8cc477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:37.901111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-ca0d179a-5893-4ab4-ab4d-f5e02d8cc477 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:37.902310 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ca0d179a-5893-4ab4-ab4d-f5e02d8cc477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:37.907734 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d118c2d-d98d-4c04-a3ec-d7938f635f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:37.908418 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d118c2d-d98d-4c04-a3ec-d7938f635f0b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:37.908708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d118c2d-d98d-4c04-a3ec-d7938f635f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:37.908990 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d118c2d-d98d-4c04-a3ec-d7938f635f0b None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:37.909745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d118c2d-d98d-4c04-a3ec-d7938f635f0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:37.910060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d118c2d-d98d-4c04-a3ec-d7938f635f0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:37.911472 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d118c2d-d98d-4c04-a3ec-d7938f635f0b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:37.913433 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:37.919979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 89/181] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:37.921328 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5febd9-e6c3-4818-93e3-7124dda3aaed None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:37.921622 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5febd9-e6c3-4818-93e3-7124dda3aaed None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:37.921908 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5febd9-e6c3-4818-93e3-7124dda3aaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:37.937551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2b5febd9-e6c3-4818-93e3-7124dda3aaed None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:37.938756 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2b5febd9-e6c3-4818-93e3-7124dda3aaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:37.970199 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 93/182] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:37.971523 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fde95b-2fca-405c-8833-70b7f8d78531 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:37.971821 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fde95b-2fca-405c-8833-70b7f8d78531 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:37.972101 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fde95b-2fca-405c-8833-70b7f8d78531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:37.987723 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-b1fde95b-2fca-405c-8833-70b7f8d78531 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:37.988911 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b1fde95b-2fca-405c-8833-70b7f8d78531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:38.005678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 90/183] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:38.006514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5febd9-e6c3-4818-93e3-7124dda3aaed None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:38.055430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 94/184] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:38.056649 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fde95b-2fca-405c-8833-70b7f8d78531 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:38.084233 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:38.130865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5febd9-e6c3-4818-93e3-7124dda3aaed None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:38.134186 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:38.157774 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fde95b-2fca-405c-8833-70b7f8d78531 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:38.165547 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718290d8-50dd-4846-abd8-8934fe88409f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:38.167079 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718290d8-50dd-4846-abd8-8934fe88409f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:38.167559 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718290d8-50dd-4846-abd8-8934fe88409f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:38.168030 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718290d8-50dd-4846-abd8-8934fe88409f None admin] PATH_INFO: `/v3/users/f89c7583bd684f1a883ca3ccb2230e5b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:38.169863 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718290d8-50dd-4846-abd8-8934fe88409f None admin] RBAC: Authorizing `identity:get_user(user_id=f89c7583bd684f1a883ca3ccb2230e5b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:38.171392 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718290d8-50dd-4846-abd8-8934fe88409f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:38.173091 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 91/185] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:38 2022] GET /identity/v3/users/f89c7583bd684f1a883ca3ccb2230e5b => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:38.173858 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718290d8-50dd-4846-abd8-8934fe88409f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:38.198899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2531e-e029-4317-9f73-b89ed64000ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:38.199754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2531e-e029-4317-9f73-b89ed64000ed None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:38.200033 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2531e-e029-4317-9f73-b89ed64000ed None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:38.200304 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2531e-e029-4317-9f73-b89ed64000ed None admin] PATH_INFO: `/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:38.202038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2531e-e029-4317-9f73-b89ed64000ed None admin] RBAC: Authorizing `identity:get_project(project_id=e3f26f512b354f6ebdbf6c8d3cc3e472)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:38.203803 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2531e-e029-4317-9f73-b89ed64000ed None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:38.205484 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 95/186] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:38 2022] GET /identity/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:38.206237 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2531e-e029-4317-9f73-b89ed64000ed None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:38.282002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718290d8-50dd-4846-abd8-8934fe88409f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:38.302148 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2531e-e029-4317-9f73-b89ed64000ed None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:38.306369 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:38.315831 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e9e379-387f-4db5-ad69-b5511d6a1bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:38.316706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9e379-387f-4db5-ad69-b5511d6a1bdf None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:38.316990 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9e379-387f-4db5-ad69-b5511d6a1bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:38.317272 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9e379-387f-4db5-ad69-b5511d6a1bdf None admin] PATH_INFO: `/v3/users/6bb9868c931f4584aaa5206e6a324763` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:38.332711 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e9e379-387f-4db5-ad69-b5511d6a1bdf None admin] RBAC: Authorizing `identity:get_user(user_id=6bb9868c931f4584aaa5206e6a324763)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:38.334161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e9e379-387f-4db5-ad69-b5511d6a1bdf None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:38.340345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 92/187] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:38 2022] GET /identity/v3/users/6bb9868c931f4584aaa5206e6a324763 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:38.340345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e9e379-387f-4db5-ad69-b5511d6a1bdf None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:38.340561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff650f6-e7c7-4ee7-ab2d-d06df6b8807f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:38.340561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff650f6-e7c7-4ee7-ab2d-d06df6b8807f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:38.340561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff650f6-e7c7-4ee7-ab2d-d06df6b8807f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:38.340561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff650f6-e7c7-4ee7-ab2d-d06df6b8807f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:38.340854 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff650f6-e7c7-4ee7-ab2d-d06df6b8807f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:38.340854 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff650f6-e7c7-4ee7-ab2d-d06df6b8807f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:38.346375 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 96/188] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:24:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:38.346375 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff650f6-e7c7-4ee7-ab2d-d06df6b8807f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:38.356837 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:38.441085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff650f6-e7c7-4ee7-ab2d-d06df6b8807f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:38.442341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e9e379-387f-4db5-ad69-b5511d6a1bdf None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:38.476114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4b283-e3ff-4484-9dfe-7f24f1af8213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:38.476302 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273fcc5e-0fa6-4416-a029-94f666dc62c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:38.476911 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4b283-e3ff-4484-9dfe-7f24f1af8213 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:38.477075 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fcc5e-0fa6-4416-a029-94f666dc62c5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:38.477194 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4b283-e3ff-4484-9dfe-7f24f1af8213 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:38.477355 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fcc5e-0fa6-4416-a029-94f666dc62c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:38.477466 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4b283-e3ff-4484-9dfe-7f24f1af8213 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:38.477628 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fcc5e-0fa6-4416-a029-94f666dc62c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:38.488217 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273fcc5e-0fa6-4416-a029-94f666dc62c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:38.488331 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb4b283-e3ff-4484-9dfe-7f24f1af8213 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:38.496009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-aeb4b283-e3ff-4484-9dfe-7f24f1af8213 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 31 14:24:38.496009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:38.496009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:38.496009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:38.496009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:38.496009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:38.496009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:38.496009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:38.496009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:38.496009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:38.496009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:38.496009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:38.496009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:38.496009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:38.496009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:38.496009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:38.497042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:38.497042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:38.497042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:38.497042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:38.497042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:38.497042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:38.497042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:38.497042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:38.497042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:38.497042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:38.497042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:38.497042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:24:38.497042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:38.497042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:38.497042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:38.497042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:38.498667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:38.498667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:38.498667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:38.498667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:38.498667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:38.498667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:38.498667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:38.498667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:38.498667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:38.498667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:38.498667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 31 14:24:38.498667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:24:38.499795 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-273fcc5e-0fa6-4416-a029-94f666dc62c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:24:38.499795 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:38.499795 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:38.499795 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:38.499795 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:38.499795 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:38.499795 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:38.499795 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:38.499795 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:38.499795 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:38.499795 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:38.499795 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:38.499795 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:38.499795 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:38.499795 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:38.499795 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:38.501968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:38.501968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:38.501968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:38.501968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:38.501968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:38.501968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:38.501968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:38.501968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:38.501968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:38.501968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:38.501968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:38.501968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:24:38.501968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:38.501968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:38.501968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:38.501968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:38.504199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:38.504199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:38.504199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:38.504199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:38.504199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:38.504199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:38.504199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:38.504199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:38.504199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:38.504199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:38.504199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:24:38.504199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:24:38.505074 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 97/189] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:38.505155 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 93/190] 10.222.0.67 () {56 vars in 1114 bytes} [Tue May 31 14:24:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:38.505155 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273fcc5e-0fa6-4416-a029-94f666dc62c5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:38.505297 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4b283-e3ff-4484-9dfe-7f24f1af8213 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:38.548914 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:38.571217 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:38.600707 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4b283-e3ff-4484-9dfe-7f24f1af8213 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:38.603016 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273fcc5e-0fa6-4416-a029-94f666dc62c5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:38.635066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a8841-98bf-472f-959b-4485115ee331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:38.635714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a8841-98bf-472f-959b-4485115ee331 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:38.636233 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083d804-e512-4d53-bccc-83569797f4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:38.636336 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a8841-98bf-472f-959b-4485115ee331 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:38.636336 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a8841-98bf-472f-959b-4485115ee331 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:38.636497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083d804-e512-4d53-bccc-83569797f4ac None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:38.636755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083d804-e512-4d53-bccc-83569797f4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:38.637040 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083d804-e512-4d53-bccc-83569797f4ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:38.637878 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6083d804-e512-4d53-bccc-83569797f4ac None admin] RBAC: Adding query filter params (role.id=e8d679596ffb45939229d1e2a13ce9b0, scope.project.id=e3f26f512b354f6ebdbf6c8d3cc3e472, user.id=c01eab2db7454d578ce76aa7d022d460) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:38.638195 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6083d804-e512-4d53-bccc-83569797f4ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:38.639683 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6083d804-e512-4d53-bccc-83569797f4ac None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:38.645520 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 94/191] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:24:38 2022] GET /identity/v3/role_assignments?user.id=c01eab2db7454d578ce76aa7d022d460&scope.project.id=e3f26f512b354f6ebdbf6c8d3cc3e472&role.id=e8d679596ffb45939229d1e2a13ce9b0 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:38.646296 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083d804-e512-4d53-bccc-83569797f4ac None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:38.710406 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a8841-98bf-472f-959b-4485115ee331 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:38.711970 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a8841-98bf-472f-959b-4485115ee331 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:38.770862 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083d804-e512-4d53-bccc-83569797f4ac None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:38.781302 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:38.787223 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:38.820213 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e5132-0a69-448a-b66a-9294c0c193f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:38.820882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e5132-0a69-448a-b66a-9294c0c193f5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:38.821175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e5132-0a69-448a-b66a-9294c0c193f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:38.821506 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e5132-0a69-448a-b66a-9294c0c193f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:38.826558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 98/192] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:24:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:24:38.826558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a8841-98bf-472f-959b-4485115ee331 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:38.913181 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e5132-0a69-448a-b66a-9294c0c193f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:38.915271 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e5132-0a69-448a-b66a-9294c0c193f5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:38.936173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a8841-98bf-472f-959b-4485115ee331 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:38.970609 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9dd11-32f4-44c9-af4f-b2b1434041a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:38.971311 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9dd11-32f4-44c9-af4f-b2b1434041a8 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:38.971595 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9dd11-32f4-44c9-af4f-b2b1434041a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:38.971869 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9dd11-32f4-44c9-af4f-b2b1434041a8 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:38.972588 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b9dd11-32f4-44c9-af4f-b2b1434041a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:38.974041 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b9dd11-32f4-44c9-af4f-b2b1434041a8 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:38.993465 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 99/193] 10.222.0.67 () {60 vars in 1132 bytes} [Tue May 31 14:24:38 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:24:38.994276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9dd11-32f4-44c9-af4f-b2b1434041a8 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:39.019904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 95/194] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:24:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:24:39.020789 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e5132-0a69-448a-b66a-9294c0c193f5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:39.032411 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:39.035158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:39.113130 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9dd11-32f4-44c9-af4f-b2b1434041a8 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:39.143160 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e5132-0a69-448a-b66a-9294c0c193f5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:39.152412 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5068a-035b-41b3-a349-7710d64b9a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:39.153101 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5068a-035b-41b3-a349-7710d64b9a70 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:39.153386 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5068a-035b-41b3-a349-7710d64b9a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:39.154217 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5068a-035b-41b3-a349-7710d64b9a70 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:39.155048 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f5068a-035b-41b3-a349-7710d64b9a70 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:39.155357 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f5068a-035b-41b3-a349-7710d64b9a70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:39.156770 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f5068a-035b-41b3-a349-7710d64b9a70 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:39.166676 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 100/195] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:24:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:39.167465 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5068a-035b-41b3-a349-7710d64b9a70 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:39.184645 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3109c-55fe-4e06-9976-abf820ccec59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:39.185900 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3109c-55fe-4e06-9976-abf820ccec59 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:39.186189 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3109c-55fe-4e06-9976-abf820ccec59 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:39.186473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3109c-55fe-4e06-9976-abf820ccec59 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:39.187267 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c3109c-55fe-4e06-9976-abf820ccec59 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:39.187584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c3109c-55fe-4e06-9976-abf820ccec59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:39.189006 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c3109c-55fe-4e06-9976-abf820ccec59 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:39.198082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 96/196] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:24:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:39.198872 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3109c-55fe-4e06-9976-abf820ccec59 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:39.282406 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:39.283994 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:39.295507 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5068a-035b-41b3-a349-7710d64b9a70 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:39.325644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3109c-55fe-4e06-9976-abf820ccec59 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:39.339448 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1b5a5-e08e-424b-9702-64197198e9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:39.340343 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1b5a5-e08e-424b-9702-64197198e9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:39.340626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1b5a5-e08e-424b-9702-64197198e9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:39.340905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1b5a5-e08e-424b-9702-64197198e9d1 None admin] PATH_INFO: `/v3/projects/1aac711712614ffd98b5667d5dc8bc91` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:39.342185 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1b5a5-e08e-424b-9702-64197198e9d1 None admin] RBAC: Authorizing `identity:get_project(project_id=1aac711712614ffd98b5667d5dc8bc91)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:39.343691 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1b5a5-e08e-424b-9702-64197198e9d1 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:39.345411 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 101/197] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:39 2022] GET /identity/v3/projects/1aac711712614ffd98b5667d5dc8bc91 => generated 312 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:39.346721 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1b5a5-e08e-424b-9702-64197198e9d1 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:39.372969 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d5ec7-1501-4162-b1d5-bed6643241b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:39.374586 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d5ec7-1501-4162-b1d5-bed6643241b8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:39.374943 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d5ec7-1501-4162-b1d5-bed6643241b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:39.375515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d5ec7-1501-4162-b1d5-bed6643241b8 None admin] PATH_INFO: `/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:39.376987 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5d5ec7-1501-4162-b1d5-bed6643241b8 None admin] RBAC: Authorizing `identity:get_project(project_id=bef8d3a846e34a8daf7eecb46a5fd2f8)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:39.379203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5d5ec7-1501-4162-b1d5-bed6643241b8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:39.381890 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 97/198] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:39 2022] GET /identity/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:39.382694 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d5ec7-1501-4162-b1d5-bed6643241b8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:39.462295 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1b5a5-e08e-424b-9702-64197198e9d1 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:39.502775 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4902b-8439-4744-9c71-93bd00936d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:39.503664 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4902b-8439-4744-9c71-93bd00936d5a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:39.503950 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4902b-8439-4744-9c71-93bd00936d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:39.504222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4902b-8439-4744-9c71-93bd00936d5a None admin] PATH_INFO: `/v3/users/6bb9868c931f4584aaa5206e6a324763` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:39.505429 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4902b-8439-4744-9c71-93bd00936d5a None admin] RBAC: Authorizing `identity:get_user(user_id=6bb9868c931f4584aaa5206e6a324763)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:39.506933 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4902b-8439-4744-9c71-93bd00936d5a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:39.508591 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 102/199] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:39 2022] GET /identity/v3/users/6bb9868c931f4584aaa5206e6a324763 => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:39.508904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d5ec7-1501-4162-b1d5-bed6643241b8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:39.509356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4902b-8439-4744-9c71-93bd00936d5a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:39.542183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:39.552586 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc48746-268a-4ec3-a84b-b1671974b905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:39.553481 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc48746-268a-4ec3-a84b-b1671974b905 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:39.553767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc48746-268a-4ec3-a84b-b1671974b905 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:39.554047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc48746-268a-4ec3-a84b-b1671974b905 None admin] PATH_INFO: `/v3/users/2d38ae56e517482e86864dab992d1c22` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:39.555568 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc48746-268a-4ec3-a84b-b1671974b905 None admin] RBAC: Authorizing `identity:get_user(user_id=2d38ae56e517482e86864dab992d1c22)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:39.557933 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc48746-268a-4ec3-a84b-b1671974b905 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:39.561052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 98/200] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:39 2022] GET /identity/v3/users/2d38ae56e517482e86864dab992d1c22 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:39.562346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc48746-268a-4ec3-a84b-b1671974b905 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:39.573935 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:39.626125 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4902b-8439-4744-9c71-93bd00936d5a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:39.664606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20ef95-4855-4e56-ba04-f681acb67546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:39.665347 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20ef95-4855-4e56-ba04-f681acb67546 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:39.665600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20ef95-4855-4e56-ba04-f681acb67546 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:39.665837 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20ef95-4855-4e56-ba04-f681acb67546 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:39.666712 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a20ef95-4855-4e56-ba04-f681acb67546 None admin] RBAC: Adding query filter params (role.id=9ce8fdecf52d482d894fdeec4f530624, scope.project.id=1aac711712614ffd98b5667d5dc8bc91, user.id=f89c7583bd684f1a883ca3ccb2230e5b) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:39.667037 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a20ef95-4855-4e56-ba04-f681acb67546 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:39.668494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a20ef95-4855-4e56-ba04-f681acb67546 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:39.678748 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 103/201] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:24:39 2022] GET /identity/v3/role_assignments?user.id=f89c7583bd684f1a883ca3ccb2230e5b&scope.project.id=1aac711712614ffd98b5667d5dc8bc91&role.id=9ce8fdecf52d482d894fdeec4f530624 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:39.679722 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20ef95-4855-4e56-ba04-f681acb67546 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:39.708061 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc48746-268a-4ec3-a84b-b1671974b905 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:39.790123 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f30dba-6339-47fc-91f5-62f493439fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:39.790751 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f30dba-6339-47fc-91f5-62f493439fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:39.791047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f30dba-6339-47fc-91f5-62f493439fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:39.791329 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f30dba-6339-47fc-91f5-62f493439fd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:39.792183 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f30dba-6339-47fc-91f5-62f493439fd9 None admin] RBAC: Adding query filter params (role.id=e8d679596ffb45939229d1e2a13ce9b0, scope.project.id=bef8d3a846e34a8daf7eecb46a5fd2f8, user.id=6bb9868c931f4584aaa5206e6a324763) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:39.792505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f30dba-6339-47fc-91f5-62f493439fd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:39.793964 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f30dba-6339-47fc-91f5-62f493439fd9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:39.805455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:39.816765 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20ef95-4855-4e56-ba04-f681acb67546 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:39.817366 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 99/202] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:24:39 2022] GET /identity/v3/role_assignments?user.id=6bb9868c931f4584aaa5206e6a324763&scope.project.id=bef8d3a846e34a8daf7eecb46a5fd2f8&role.id=e8d679596ffb45939229d1e2a13ce9b0 => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:39.818068 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f30dba-6339-47fc-91f5-62f493439fd9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:39.865434 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9babd707-8317-41d7-813c-ec148b24b036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:39.866141 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9babd707-8317-41d7-813c-ec148b24b036 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:39.866427 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9babd707-8317-41d7-813c-ec148b24b036 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:39.868698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9babd707-8317-41d7-813c-ec148b24b036 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:39.959943 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:39.989730 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9babd707-8317-41d7-813c-ec148b24b036 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:39.991580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9babd707-8317-41d7-813c-ec148b24b036 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:40.025618 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f30dba-6339-47fc-91f5-62f493439fd9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:40.093361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc92ce0-4b2e-4aa6-bb9c-2965969cca83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:40.093949 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc92ce0-4b2e-4aa6-bb9c-2965969cca83 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:40.094238 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc92ce0-4b2e-4aa6-bb9c-2965969cca83 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:40.094522 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc92ce0-4b2e-4aa6-bb9c-2965969cca83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:40.170008 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:40.183111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 104/203] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:24:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:24:40.188954 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9babd707-8317-41d7-813c-ec148b24b036 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:40.196847 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc92ce0-4b2e-4aa6-bb9c-2965969cca83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:40.198699 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc92ce0-4b2e-4aa6-bb9c-2965969cca83 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:40.278670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:40.296303 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9babd707-8317-41d7-813c-ec148b24b036 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:40.331638 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982e8df-0ab0-4a58-bb25-e4303f3f2b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:40.332102 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 100/204] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:24:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:24:40.332704 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982e8df-0ab0-4a58-bb25-e4303f3f2b8d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:40.332989 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982e8df-0ab0-4a58-bb25-e4303f3f2b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:40.333263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982e8df-0ab0-4a58-bb25-e4303f3f2b8d None admin] PATH_INFO: `/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:40.333814 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc92ce0-4b2e-4aa6-bb9c-2965969cca83 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:40.334601 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9982e8df-0ab0-4a58-bb25-e4303f3f2b8d None admin] RBAC: Authorizing `identity:get_project(project_id=bef8d3a846e34a8daf7eecb46a5fd2f8)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:40.336254 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9982e8df-0ab0-4a58-bb25-e4303f3f2b8d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:40.338098 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 105/205] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:40 2022] GET /identity/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:40.345193 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982e8df-0ab0-4a58-bb25-e4303f3f2b8d None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:40.423203 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:40.468677 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982e8df-0ab0-4a58-bb25-e4303f3f2b8d None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:40.486813 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc92ce0-4b2e-4aa6-bb9c-2965969cca83 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:40.508272 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddfef3-9128-4c53-b1fa-f9ae2fd32731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:40.509404 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddfef3-9128-4c53-b1fa-f9ae2fd32731 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:40.509890 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddfef3-9128-4c53-b1fa-f9ae2fd32731 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:40.510365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddfef3-9128-4c53-b1fa-f9ae2fd32731 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:40.511565 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ddfef3-9128-4c53-b1fa-f9ae2fd32731 None admin] RBAC: Adding query filter params (role.id=1b9c18d4b8b540bf9825080748cec3bc, scope.project.id=bef8d3a846e34a8daf7eecb46a5fd2f8, user.id=6bb9868c931f4584aaa5206e6a324763) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:40.512127 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ddfef3-9128-4c53-b1fa-f9ae2fd32731 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:40.513930 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ddfef3-9128-4c53-b1fa-f9ae2fd32731 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:40.520692 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 106/206] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:24:40 2022] GET /identity/v3/role_assignments?user.id=6bb9868c931f4584aaa5206e6a324763&scope.project.id=bef8d3a846e34a8daf7eecb46a5fd2f8&role.id=1b9c18d4b8b540bf9825080748cec3bc => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:40.554016 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3901c-e6d6-4d89-bedd-5e9311183d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:40.554957 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3901c-e6d6-4d89-bedd-5e9311183d7b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:40.555247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3901c-e6d6-4d89-bedd-5e9311183d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:40.555528 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3901c-e6d6-4d89-bedd-5e9311183d7b None admin] PATH_INFO: `/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:40.559542 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e3901c-e6d6-4d89-bedd-5e9311183d7b None admin] RBAC: Authorizing `identity:get_project(project_id=e3f26f512b354f6ebdbf6c8d3cc3e472)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:40.561094 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e3901c-e6d6-4d89-bedd-5e9311183d7b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:40.563329 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 101/207] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:40 2022] GET /identity/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:40.568925 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddfef3-9128-4c53-b1fa-f9ae2fd32731 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:40.730481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddfef3-9128-4c53-b1fa-f9ae2fd32731 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:40.767218 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:40.771554 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb637c-b0e5-4fcc-92eb-b4c25ac0245c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:40.773283 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb637c-b0e5-4fcc-92eb-b4c25ac0245c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:40.774288 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb637c-b0e5-4fcc-92eb-b4c25ac0245c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:40.775125 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb637c-b0e5-4fcc-92eb-b4c25ac0245c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:40.776344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb637c-b0e5-4fcc-92eb-b4c25ac0245c None admin] RBAC: Adding query filter params (role.id=8655041bde1948a1be442c6239496c22, scope.project.id=e3f26f512b354f6ebdbf6c8d3cc3e472, user.id=2d38ae56e517482e86864dab992d1c22) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:40.776881 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb637c-b0e5-4fcc-92eb-b4c25ac0245c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:40.778531 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb637c-b0e5-4fcc-92eb-b4c25ac0245c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:40.785905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 107/208] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:24:40 2022] GET /identity/v3/role_assignments?user.id=2d38ae56e517482e86864dab992d1c22&scope.project.id=e3f26f512b354f6ebdbf6c8d3cc3e472&role.id=8655041bde1948a1be442c6239496c22 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:41.780497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3c116-efda-468c-aca2-184b3ce6f445 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:41.781023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3c116-efda-468c-aca2-184b3ce6f445 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:41.781342 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3c116-efda-468c-aca2-184b3ce6f445 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:41.783159 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 102/209] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:41.788586 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbadb4b-efa4-490e-b6c9-3a8d9c38f108 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:41.789007 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbadb4b-efa4-490e-b6c9-3a8d9c38f108 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:41.789322 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbadb4b-efa4-490e-b6c9-3a8d9c38f108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:41.807534 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-2bbadb4b-efa4-490e-b6c9-3a8d9c38f108 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:41.808749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2bbadb4b-efa4-490e-b6c9-3a8d9c38f108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:41.883667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 108/210] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:41.908568 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3c116-efda-468c-aca2-184b3ce6f445 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:41.919811 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c2917-872b-4936-bd6f-931da5989e7b None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:41.920105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c2917-872b-4936-bd6f-931da5989e7b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:41.920386 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c2917-872b-4936-bd6f-931da5989e7b None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:41.921931 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 109/211] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:41.927251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561791d-a4d6-4cfd-9646-e5a9339a65a2 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:41.927835 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561791d-a4d6-4cfd-9646-e5a9339a65a2 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:41.927835 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561791d-a4d6-4cfd-9646-e5a9339a65a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:41.944768 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6561791d-a4d6-4cfd-9646-e5a9339a65a2 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:41.945971 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6561791d-a4d6-4cfd-9646-e5a9339a65a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:41.955839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:42.026158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3c116-efda-468c-aca2-184b3ce6f445 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:42.038826 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 110/212] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 31 14:24:42.058337 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561791d-a4d6-4cfd-9646-e5a9339a65a2 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:42.063892 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fe8d9-9d5b-4234-a099-5675d5beb0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:42.064698 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fe8d9-9d5b-4234-a099-5675d5beb0e0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:42.064984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fe8d9-9d5b-4234-a099-5675d5beb0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:42.065257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fe8d9-9d5b-4234-a099-5675d5beb0e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:42.076571 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3fe8d9-9d5b-4234-a099-5675d5beb0e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:42.084651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-8c3fe8d9-9d5b-4234-a099-5675d5beb0e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 31 14:24:42.084651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:42.084651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:42.084651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:42.084651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:42.084651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:42.084651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:42.084651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:42.084651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:42.084651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:42.084651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:42.084651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:42.084651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:42.084651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:42.084651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:42.084651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:42.085515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:42.085515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:42.085515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:42.085515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:42.085515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:42.085515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:42.085515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:42.085515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:42.085515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:42.085515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:42.085515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:42.085515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:24:42.085515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:42.085515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:42.085515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:42.085515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:42.086304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:42.086304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:42.086304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:42.086304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:42.086304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:42.086304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:42.086304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:42.086304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:42.086304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:42.086304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:42.086304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 31 14:24:42.086304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:24:42.086304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 103/213] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:42.093096 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fe8d9-9d5b-4234-a099-5675d5beb0e0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:42.120167 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:42.186299 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561791d-a4d6-4cfd-9646-e5a9339a65a2 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:42.202704 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:42.208096 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fe8d9-9d5b-4234-a099-5675d5beb0e0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:42.227710 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2107cc-3e5a-4775-aa99-6c2ca0aacdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:42.228516 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2107cc-3e5a-4775-aa99-6c2ca0aacdfc None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:42.228801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2107cc-3e5a-4775-aa99-6c2ca0aacdfc None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:42.229079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2107cc-3e5a-4775-aa99-6c2ca0aacdfc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:42.230234 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2107cc-3e5a-4775-aa99-6c2ca0aacdfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:42.232274 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2107cc-3e5a-4775-aa99-6c2ca0aacdfc None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:42.233951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 111/214] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:24:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:42.239397 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2107cc-3e5a-4775-aa99-6c2ca0aacdfc None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:42.252176 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce947227-f7ef-4bd4-a0e2-c89457cfffb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:42.253292 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce947227-f7ef-4bd4-a0e2-c89457cfffb3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:42.253735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce947227-f7ef-4bd4-a0e2-c89457cfffb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:42.254143 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce947227-f7ef-4bd4-a0e2-c89457cfffb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:42.254943 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce947227-f7ef-4bd4-a0e2-c89457cfffb3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:42.255289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce947227-f7ef-4bd4-a0e2-c89457cfffb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:42.257513 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce947227-f7ef-4bd4-a0e2-c89457cfffb3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:42.269095 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 104/215] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:24:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:24:42.274567 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce947227-f7ef-4bd4-a0e2-c89457cfffb3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:42.348897 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2107cc-3e5a-4775-aa99-6c2ca0aacdfc None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:42.364851 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:42.380352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce947227-f7ef-4bd4-a0e2-c89457cfffb3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:42.388793 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13162bd-2973-43a3-a9f5-9a150d7d2c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:42.389472 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13162bd-2973-43a3-a9f5-9a150d7d2c26 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:42.389754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13162bd-2973-43a3-a9f5-9a150d7d2c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:42.390028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13162bd-2973-43a3-a9f5-9a150d7d2c26 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:42.391152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13162bd-2973-43a3-a9f5-9a150d7d2c26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:42.392930 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13162bd-2973-43a3-a9f5-9a150d7d2c26 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:42.414711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 112/216] 10.222.0.67 () {60 vars in 1132 bytes} [Tue May 31 14:24:42 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:24:42.418725 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586b76f-0748-45a5-a837-4f239114f96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:42.419856 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586b76f-0748-45a5-a837-4f239114f96c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:42.420345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586b76f-0748-45a5-a837-4f239114f96c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:42.420762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586b76f-0748-45a5-a837-4f239114f96c None admin] PATH_INFO: `/v3/users/822895f606f24a74ad77852b46c49434` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:42.425891 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9a914-2e53-4db9-b823-ba24f505a3c3 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:42.426311 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9a914-2e53-4db9-b823-ba24f505a3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:42.426731 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9a914-2e53-4db9-b823-ba24f505a3c3 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:42.432506 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 113/217] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:42 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:42.434239 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d98125-ccbd-497f-88c1-5a6b3b91c15e None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:42.434536 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d98125-ccbd-497f-88c1-5a6b3b91c15e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:42.436655 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d98125-ccbd-497f-88c1-5a6b3b91c15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:42.445244 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b586b76f-0748-45a5-a837-4f239114f96c None admin] RBAC: Authorizing `identity:get_user(user_id=822895f606f24a74ad77852b46c49434)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:42.446805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b586b76f-0748-45a5-a837-4f239114f96c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:42.448670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 105/218] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:42 2022] GET /identity/v3/users/822895f606f24a74ad77852b46c49434 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:24:42.455527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586b76f-0748-45a5-a837-4f239114f96c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:42.458373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-79d98125-ccbd-497f-88c1-5a6b3b91c15e None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:42.460734 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-79d98125-ccbd-497f-88c1-5a6b3b91c15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:42.478168 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:42.547132 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 114/219] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:42.565975 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d98125-ccbd-497f-88c1-5a6b3b91c15e None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:42.574212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586b76f-0748-45a5-a837-4f239114f96c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:42.615700 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad92905-5ad2-46fb-a349-350c1e412366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:42.616629 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad92905-5ad2-46fb-a349-350c1e412366 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:42.617054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad92905-5ad2-46fb-a349-350c1e412366 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:42.617481 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad92905-5ad2-46fb-a349-350c1e412366 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:42.679520 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:42.683866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d98125-ccbd-497f-88c1-5a6b3b91c15e None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:42.713774 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad92905-5ad2-46fb-a349-350c1e412366 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:42.716022 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad92905-5ad2-46fb-a349-350c1e412366 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:42.736428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8b6fa-935a-4eaf-b301-77db52e6452e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:42.736428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8b6fa-935a-4eaf-b301-77db52e6452e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:42.736428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8b6fa-935a-4eaf-b301-77db52e6452e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:42.736428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8b6fa-935a-4eaf-b301-77db52e6452e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:42.741565 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:42.753205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea8b6fa-935a-4eaf-b301-77db52e6452e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:42.764563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-5ea8b6fa-935a-4eaf-b301-77db52e6452e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:24:42.764563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:42.764563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:42.764563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:42.764563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:42.764563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:42.764563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:42.764563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:42.764563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:42.764563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:42.764563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:42.764563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:42.764563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:42.764563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:42.764563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:42.764563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:42.765467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:42.765467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:42.765467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:42.765467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:42.765467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:42.765467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:42.765467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:42.765467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:42.765467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:42.765467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:42.765467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:42.765467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:24:42.765467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:42.765467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:42.765467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:42.765467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:42.766263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:42.766263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:42.766263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:42.766263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:42.766263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:42.766263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:42.766263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:42.766263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:42.766263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:42.766263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:42.766263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:24:42.766263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:24:42.769305 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 115/220] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:24:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:42.775262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8b6fa-935a-4eaf-b301-77db52e6452e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:42.854110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 106/221] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:24:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 31 14:24:42.863233 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad92905-5ad2-46fb-a349-350c1e412366 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:42.927761 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8b6fa-935a-4eaf-b301-77db52e6452e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:42.973220 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad255ed6-7860-4aca-9f5b-3f9f00f733ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:42.973911 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad255ed6-7860-4aca-9f5b-3f9f00f733ca None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:42.974201 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad255ed6-7860-4aca-9f5b-3f9f00f733ca None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:42.974879 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad255ed6-7860-4aca-9f5b-3f9f00f733ca None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:42.975931 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad255ed6-7860-4aca-9f5b-3f9f00f733ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:42.976303 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad255ed6-7860-4aca-9f5b-3f9f00f733ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:42.978391 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad255ed6-7860-4aca-9f5b-3f9f00f733ca None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:42.984044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad92905-5ad2-46fb-a349-350c1e412366 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:42.989823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 116/222] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:42.995330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad255ed6-7860-4aca-9f5b-3f9f00f733ca None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:43.008378 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:43.023838 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f22cde-4da5-4db1-a517-317840043598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:43.024874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f22cde-4da5-4db1-a517-317840043598 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:43.025261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f22cde-4da5-4db1-a517-317840043598 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:43.025639 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f22cde-4da5-4db1-a517-317840043598 None admin] PATH_INFO: `/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:43.026977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f22cde-4da5-4db1-a517-317840043598 None admin] RBAC: Authorizing `identity:get_project(project_id=bef8d3a846e34a8daf7eecb46a5fd2f8)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:43.028576 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f22cde-4da5-4db1-a517-317840043598 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:43.030739 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 107/223] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:42 2022] GET /identity/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:43.031197 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:43.051682 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f22cde-4da5-4db1-a517-317840043598 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:43.124743 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad255ed6-7860-4aca-9f5b-3f9f00f733ca None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:43.173089 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d50a2e-6317-4b4c-8814-d08ce0e001ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:43.173996 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d50a2e-6317-4b4c-8814-d08ce0e001ff None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:43.174282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d50a2e-6317-4b4c-8814-d08ce0e001ff None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:43.174562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d50a2e-6317-4b4c-8814-d08ce0e001ff None admin] PATH_INFO: `/v3/users/6bb9868c931f4584aaa5206e6a324763` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:43.175838 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d50a2e-6317-4b4c-8814-d08ce0e001ff None admin] RBAC: Authorizing `identity:get_user(user_id=6bb9868c931f4584aaa5206e6a324763)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:43.177853 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d50a2e-6317-4b4c-8814-d08ce0e001ff None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:43.180247 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 117/224] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:42 2022] GET /identity/v3/users/6bb9868c931f4584aaa5206e6a324763 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:43.182278 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31855702-91ee-4474-8b95-ce8e06fd2244 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:43.182571 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31855702-91ee-4474-8b95-ce8e06fd2244 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:43.182878 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31855702-91ee-4474-8b95-ce8e06fd2244 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:43.185530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 118/225] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:43.187478 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31855702-91ee-4474-8b95-ce8e06fd2244 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:43.210677 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f22cde-4da5-4db1-a517-317840043598 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:43.256883 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825b729-5318-4c77-98ed-f6bc334b6827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:43.257916 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825b729-5318-4c77-98ed-f6bc334b6827 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:43.258260 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825b729-5318-4c77-98ed-f6bc334b6827 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:43.258549 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825b729-5318-4c77-98ed-f6bc334b6827 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:43.259942 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825b729-5318-4c77-98ed-f6bc334b6827 None admin] RBAC: Adding query filter params (role.id=9ce8fdecf52d482d894fdeec4f530624, scope.project.id=bef8d3a846e34a8daf7eecb46a5fd2f8, user.id=822895f606f24a74ad77852b46c49434) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:43.260265 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825b729-5318-4c77-98ed-f6bc334b6827 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:43.261788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825b729-5318-4c77-98ed-f6bc334b6827 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:43.269052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 108/226] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:24:43 2022] GET /identity/v3/role_assignments?user.id=822895f606f24a74ad77852b46c49434&scope.project.id=bef8d3a846e34a8daf7eecb46a5fd2f8&role.id=9ce8fdecf52d482d894fdeec4f530624 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:43.270288 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1da461-4afe-49fb-b794-a2cc2b3d53b7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:43.270579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1da461-4afe-49fb-b794-a2cc2b3d53b7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:43.270895 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1da461-4afe-49fb-b794-a2cc2b3d53b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:43.313160 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ce1da461-4afe-49fb-b794-a2cc2b3d53b7 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:43.314412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ce1da461-4afe-49fb-b794-a2cc2b3d53b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:43.339122 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:43.351803 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31855702-91ee-4474-8b95-ce8e06fd2244 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:43.396856 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5718b-a1d0-4206-ba1f-0353ec4af9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:43.397855 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5718b-a1d0-4206-ba1f-0353ec4af9ad None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:43.398208 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5718b-a1d0-4206-ba1f-0353ec4af9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:43.398556 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5718b-a1d0-4206-ba1f-0353ec4af9ad None admin] PATH_INFO: `/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:43.399913 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5718b-a1d0-4206-ba1f-0353ec4af9ad None admin] RBAC: Authorizing `identity:get_project(project_id=bef8d3a846e34a8daf7eecb46a5fd2f8)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:43.401475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5718b-a1d0-4206-ba1f-0353ec4af9ad None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:43.404148 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 119/227] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:43 2022] GET /identity/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:43.412245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:43.413609 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5718b-a1d0-4206-ba1f-0353ec4af9ad None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:43.437102 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 109/228] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:43.500813 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1da461-4afe-49fb-b794-a2cc2b3d53b7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:43.560794 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5718b-a1d0-4206-ba1f-0353ec4af9ad None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:43.604013 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddffbc-4eda-467e-9964-5a2d83155c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:43.604913 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddffbc-4eda-467e-9964-5a2d83155c34 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:43.605625 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddffbc-4eda-467e-9964-5a2d83155c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:43.605920 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddffbc-4eda-467e-9964-5a2d83155c34 None admin] PATH_INFO: `/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8/users/6bb9868c931f4584aaa5206e6a324763/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:43.608941 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dddffbc-4eda-467e-9964-5a2d83155c34 None admin] RBAC: Authorizing `identity:create_grant(project_id=bef8d3a846e34a8daf7eecb46a5fd2f8, user_id=6bb9868c931f4584aaa5206e6a324763, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:43.611051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dddffbc-4eda-467e-9964-5a2d83155c34 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:43.618451 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1da461-4afe-49fb-b794-a2cc2b3d53b7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:43.639619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 120/229] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:24:43 2022] PUT /identity/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8/users/6bb9868c931f4584aaa5206e6a324763/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:24:43.662305 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14ca31-d1fb-4dee-8cc9-11472315e782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:43.663167 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14ca31-d1fb-4dee-8cc9-11472315e782 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:43.663505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14ca31-d1fb-4dee-8cc9-11472315e782 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:43.663789 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14ca31-d1fb-4dee-8cc9-11472315e782 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:43.687996 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14ca31-d1fb-4dee-8cc9-11472315e782 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:43.696812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-ed14ca31-d1fb-4dee-8cc9-11472315e782 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:24:43.696812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:43.696812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:43.696812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:43.696812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:43.696812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:43.696812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:43.696812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:43.696812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:43.696812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:43.696812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:43.696812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:43.696812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:43.696812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:43.696812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:43.696812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:43.697784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:43.697784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:43.697784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:43.697784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:43.697784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:43.697784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:43.697784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:43.697784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:43.697784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:43.697784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:43.697784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:43.697784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:24:43.697784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:43.697784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:43.697784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:43.697784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:43.699044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:43.699044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:43.699044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:43.699044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:43.699044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:43.699044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:43.699044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:43.699044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:43.699044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:43.699044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:43.699044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:24:43.699044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:24:43.699044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 110/230] 10.222.0.67 () {56 vars in 1114 bytes} [Tue May 31 14:24:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:24:43.706089 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddffbc-4eda-467e-9964-5a2d83155c34 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:43.735957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:43.850591 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddffbc-4eda-467e-9964-5a2d83155c34 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:43.888845 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b89cf2-dad9-455b-888f-cb47a5ea361f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:43.889529 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b89cf2-dad9-455b-888f-cb47a5ea361f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:43.889812 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b89cf2-dad9-455b-888f-cb47a5ea361f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:43.890091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b89cf2-dad9-455b-888f-cb47a5ea361f None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:43.890878 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b89cf2-dad9-455b-888f-cb47a5ea361f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:43.891191 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b89cf2-dad9-455b-888f-cb47a5ea361f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:43.892583 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b89cf2-dad9-455b-888f-cb47a5ea361f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:43.902813 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 121/231] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:24:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:43.910888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14ca31-d1fb-4dee-8cc9-11472315e782 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:43.955346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:44.034380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14ca31-d1fb-4dee-8cc9-11472315e782 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:44.082011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da872ed0-b4d3-48b1-9d1e-dcacb154fd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:44.082963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da872ed0-b4d3-48b1-9d1e-dcacb154fd4d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:44.083253 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da872ed0-b4d3-48b1-9d1e-dcacb154fd4d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:44.083742 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da872ed0-b4d3-48b1-9d1e-dcacb154fd4d None admin] PATH_INFO: `/v3/users/6bb9868c931f4584aaa5206e6a324763` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:44.086846 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da872ed0-b4d3-48b1-9d1e-dcacb154fd4d None admin] RBAC: Authorizing `identity:get_user(user_id=6bb9868c931f4584aaa5206e6a324763)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:44.088359 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da872ed0-b4d3-48b1-9d1e-dcacb154fd4d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:44.090482 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 111/232] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:43 2022] GET /identity/v3/users/6bb9868c931f4584aaa5206e6a324763 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:44.096621 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b89cf2-dad9-455b-888f-cb47a5ea361f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:44.225088 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:44.246045 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b89cf2-dad9-455b-888f-cb47a5ea361f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:44.292300 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f214afa-f989-4814-b4f6-68b315cbdf8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:44.293179 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f214afa-f989-4814-b4f6-68b315cbdf8a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:44.293514 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f214afa-f989-4814-b4f6-68b315cbdf8a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:44.293799 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f214afa-f989-4814-b4f6-68b315cbdf8a None admin] PATH_INFO: `/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:44.295065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f214afa-f989-4814-b4f6-68b315cbdf8a None admin] RBAC: Authorizing `identity:get_project(project_id=bef8d3a846e34a8daf7eecb46a5fd2f8)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:44.296522 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f214afa-f989-4814-b4f6-68b315cbdf8a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:44.298407 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 122/233] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:44 2022] GET /identity/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:44.304213 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da872ed0-b4d3-48b1-9d1e-dcacb154fd4d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:44.445014 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da872ed0-b4d3-48b1-9d1e-dcacb154fd4d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:44.455239 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:44.480825 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d38ef5a-dd56-4501-9cf4-4c7204e31af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:44.482219 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38ef5a-dd56-4501-9cf4-4c7204e31af0 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:44.482510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38ef5a-dd56-4501-9cf4-4c7204e31af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:44.482873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38ef5a-dd56-4501-9cf4-4c7204e31af0 None admin] PATH_INFO: `/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8/users/6bb9868c931f4584aaa5206e6a324763/roles/1b9c18d4b8b540bf9825080748cec3bc` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:44.485257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d38ef5a-dd56-4501-9cf4-4c7204e31af0 None admin] RBAC: Authorizing `identity:create_grant(project_id=bef8d3a846e34a8daf7eecb46a5fd2f8, user_id=6bb9868c931f4584aaa5206e6a324763, role_id=1b9c18d4b8b540bf9825080748cec3bc)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:44.486774 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d38ef5a-dd56-4501-9cf4-4c7204e31af0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:44.504447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 112/234] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:24:44 2022] PUT /identity/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8/users/6bb9868c931f4584aaa5206e6a324763/roles/1b9c18d4b8b540bf9825080748cec3bc => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:24:45.377270 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a033959-c7c5-40cd-9822-2d69be112e8d None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:45.378316 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a033959-c7c5-40cd-9822-2d69be112e8d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:45.378841 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a033959-c7c5-40cd-9822-2d69be112e8d None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:45.380728 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 123/235] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:45.386411 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef1b1d-1f8f-4bfd-9035-86292be61865 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:45.387303 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef1b1d-1f8f-4bfd-9035-86292be61865 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:45.388302 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef1b1d-1f8f-4bfd-9035-86292be61865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:45.406559 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ddef1b1d-1f8f-4bfd-9035-86292be61865 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:45.408106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ddef1b1d-1f8f-4bfd-9035-86292be61865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:45.469211 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09199de9-a0f3-4134-996e-573156a6bec4 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:45.469511 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09199de9-a0f3-4134-996e-573156a6bec4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:45.469778 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09199de9-a0f3-4134-996e-573156a6bec4 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:45.471200 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 124/236] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 31 14:24:45.483974 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d7180-350a-4861-bd1e-4d00735df4f1 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:45.484262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d7180-350a-4861-bd1e-4d00735df4f1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:45.484545 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d7180-350a-4861-bd1e-4d00735df4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:45.487624 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 113/237] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:45.514129 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef1b1d-1f8f-4bfd-9035-86292be61865 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:45.532089 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-ec0d7180-350a-4861-bd1e-4d00735df4f1 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:45.535854 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ec0d7180-350a-4861-bd1e-4d00735df4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:45.650246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:45.660034 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef1b1d-1f8f-4bfd-9035-86292be61865 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:45.689651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 125/238] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 31 14:24:45.711457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22a3dc-5a88-4fbe-93fb-661675249cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:45.712641 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22a3dc-5a88-4fbe-93fb-661675249cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:45.713131 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22a3dc-5a88-4fbe-93fb-661675249cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:45.713680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22a3dc-5a88-4fbe-93fb-661675249cf4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:45.715135 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22a3dc-5a88-4fbe-93fb-661675249cf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:45.716840 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22a3dc-5a88-4fbe-93fb-661675249cf4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:45.722005 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 114/239] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:24:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:45.723290 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d7180-350a-4861-bd1e-4d00735df4f1 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:45.737593 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22a3dc-5a88-4fbe-93fb-661675249cf4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:45.826528 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:45.849613 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d7180-350a-4861-bd1e-4d00735df4f1 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:45.888736 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2025e0-e3ca-47c5-820a-5e3ac726d081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:45.889909 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2025e0-e3ca-47c5-820a-5e3ac726d081 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:45.890392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2025e0-e3ca-47c5-820a-5e3ac726d081 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:45.890927 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2025e0-e3ca-47c5-820a-5e3ac726d081 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:45.902755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2025e0-e3ca-47c5-820a-5e3ac726d081 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:45.911349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-9a2025e0-e3ca-47c5-820a-5e3ac726d081 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 31 14:24:45.911349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:45.911349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:45.911349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:45.911349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:45.911349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:45.911349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:45.911349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:45.911349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:45.911349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:45.911349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:45.911349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:45.911349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:45.911349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:45.911349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:45.911349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:45.912321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:45.912321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:45.912321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:45.912321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:45.912321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:45.912321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:45.912321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:45.912321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:45.912321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:45.912321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:45.912321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:45.912321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:24:45.912321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:45.912321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:45.912321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:45.912321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:45.913129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:45.913129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:45.913129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:45.913129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:45.913129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:45.913129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:45.913129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:45.913129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:45.913129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:45.913129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:45.913129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 31 14:24:45.913129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:24:45.915882 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 126/240] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:45.921840 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2025e0-e3ca-47c5-820a-5e3ac726d081 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:46.035732 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22a3dc-5a88-4fbe-93fb-661675249cf4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:46.036416 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2025e0-e3ca-47c5-820a-5e3ac726d081 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:46.088987 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c7a84-1412-4b69-9bcc-596b7884e8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:46.090061 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c7a84-1412-4b69-9bcc-596b7884e8ba None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:46.092071 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c7a84-1412-4b69-9bcc-596b7884e8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:46.092623 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c7a84-1412-4b69-9bcc-596b7884e8ba None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:46.093894 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c7a84-1412-4b69-9bcc-596b7884e8ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:46.096993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c7a84-1412-4b69-9bcc-596b7884e8ba None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:46.124112 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4aca7-8102-4d41-a478-7fc7f352accf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:46.128183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4aca7-8102-4d41-a478-7fc7f352accf None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:46.128828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4aca7-8102-4d41-a478-7fc7f352accf None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:46.130016 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4aca7-8102-4d41-a478-7fc7f352accf None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:46.131050 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4aca7-8102-4d41-a478-7fc7f352accf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:46.131594 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4aca7-8102-4d41-a478-7fc7f352accf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:46.133506 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4aca7-8102-4d41-a478-7fc7f352accf None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:46.139863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:46.145768 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 127/241] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:24:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:24:46.152351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4aca7-8102-4d41-a478-7fc7f352accf None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:46.157745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 115/242] 10.222.0.67 () {60 vars in 1132 bytes} [Tue May 31 14:24:45 2022] POST /identity/v3/users => generated 293 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:24:46.270917 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4aca7-8102-4d41-a478-7fc7f352accf None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:46.312386 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5626b163-4ddc-43fb-840d-d1b6f26ea541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:46.313572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626b163-4ddc-43fb-840d-d1b6f26ea541 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:46.314042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626b163-4ddc-43fb-840d-d1b6f26ea541 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:46.314548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626b163-4ddc-43fb-840d-d1b6f26ea541 None admin] PATH_INFO: `/v3/users/c41f416318cc4813877c7c3ea89bf8fd` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:46.330237 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5626b163-4ddc-43fb-840d-d1b6f26ea541 None admin] RBAC: Authorizing `identity:get_user(user_id=c41f416318cc4813877c7c3ea89bf8fd)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:46.332169 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5626b163-4ddc-43fb-840d-d1b6f26ea541 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:46.334546 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 128/243] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:46 2022] GET /identity/v3/users/c41f416318cc4813877c7c3ea89bf8fd => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:24:46.342778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c7a84-1412-4b69-9bcc-596b7884e8ba None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:46.394220 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:46.427781 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d837ba-da06-42cd-bef0-bf52d7043ca3 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:46.428747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d837ba-da06-42cd-bef0-bf52d7043ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:46.429234 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d837ba-da06-42cd-bef0-bf52d7043ca3 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:46.431261 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 129/244] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:46.440446 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c2a82-fa6f-44f6-8eae-0611e6d2b36f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:46.441040 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c2a82-fa6f-44f6-8eae-0611e6d2b36f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:46.441806 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c2a82-fa6f-44f6-8eae-0611e6d2b36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:46.465691 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-e84c2a82-fa6f-44f6-8eae-0611e6d2b36f None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:46.467491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e84c2a82-fa6f-44f6-8eae-0611e6d2b36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:46.500587 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c7a84-1412-4b69-9bcc-596b7884e8ba None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:46.545120 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f0ebd-a6e1-4bac-a7b1-3876702e9681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:46.545789 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f0ebd-a6e1-4bac-a7b1-3876702e9681 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:46.546080 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f0ebd-a6e1-4bac-a7b1-3876702e9681 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:46.546366 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f0ebd-a6e1-4bac-a7b1-3876702e9681 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:46.546908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:46.566869 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 130/245] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:46.570189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc0299-7718-451f-b2f2-8d30c82ce9f9 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:46.570519 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc0299-7718-451f-b2f2-8d30c82ce9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:46.570832 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc0299-7718-451f-b2f2-8d30c82ce9f9 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:46.572360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 131/246] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:46.577948 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946825d9-baca-4a7a-9b2b-60f6e32de354 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:46.578240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946825d9-baca-4a7a-9b2b-60f6e32de354 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:46.578519 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946825d9-baca-4a7a-9b2b-60f6e32de354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:46.602047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-946825d9-baca-4a7a-9b2b-60f6e32de354 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:46.603503 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-946825d9-baca-4a7a-9b2b-60f6e32de354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:46.634613 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f0ebd-a6e1-4bac-a7b1-3876702e9681 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:46.636198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f0ebd-a6e1-4bac-a7b1-3876702e9681 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:46.664736 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:46.695351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 132/247] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 31 14:24:46.696056 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946825d9-baca-4a7a-9b2b-60f6e32de354 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:46.784151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 116/248] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:24:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:24:46.785010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f0ebd-a6e1-4bac-a7b1-3876702e9681 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:46.849555 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946825d9-baca-4a7a-9b2b-60f6e32de354 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:46.881111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:46.919399 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a657184-3871-47a9-a79e-1157a9177197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:46.920269 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a657184-3871-47a9-a79e-1157a9177197 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:46.920559 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a657184-3871-47a9-a79e-1157a9177197 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:46.920890 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a657184-3871-47a9-a79e-1157a9177197 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:46.946679 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a657184-3871-47a9-a79e-1157a9177197 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:46.967421 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-8a657184-3871-47a9-a79e-1157a9177197 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 31 14:24:46.967421 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:46.967421 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:46.967421 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:46.967421 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:46.967421 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:46.967421 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:46.967421 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:46.967421 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:46.967421 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:46.967421 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:46.967421 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:46.967421 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:46.967421 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:46.967421 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:46.967421 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:46.968372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:46.968372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:46.968372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:46.968372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:46.968372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:46.968372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:46.968372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:46.968372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:46.968372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:46.968372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:46.968372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:46.968372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:24:46.968372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:46.968372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:46.968372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:46.968372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:46.969178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:46.969178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:46.969178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:46.969178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:46.969178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:46.969178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:46.969178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:46.969178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:46.969178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:46.969178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:46.969178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 31 14:24:46.969178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:24:46.970262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 133/249] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:46.973581 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a657184-3871-47a9-a79e-1157a9177197 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:46.975341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f0ebd-a6e1-4bac-a7b1-3876702e9681 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:47.020152 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:47.045494 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22040baf-5a39-442b-a04a-2108f2feedf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:47.046305 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22040baf-5a39-442b-a04a-2108f2feedf6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:47.046590 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22040baf-5a39-442b-a04a-2108f2feedf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:47.046890 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22040baf-5a39-442b-a04a-2108f2feedf6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:47.059323 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22040baf-5a39-442b-a04a-2108f2feedf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:47.067052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-22040baf-5a39-442b-a04a-2108f2feedf6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:24:47.067052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:47.067052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:47.067052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:47.067052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:47.067052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:47.067052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:47.067052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:47.067052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:47.067052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:47.067052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:47.067052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:47.067052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:47.067052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:47.067052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:47.067052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:47.067963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:47.067963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:47.067963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:47.067963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:47.067963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:47.067963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:47.067963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:47.067963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:47.067963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:47.067963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:47.067963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:47.067963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:24:47.067963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:47.067963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:47.067963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:47.067963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:47.068744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:47.068744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:47.068744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:47.068744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:47.068744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:47.068744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:47.068744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:47.068744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:47.068744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:47.068744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:47.068744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:24:47.068744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:24:47.068744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 117/250] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:24:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:47.069407 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22040baf-5a39-442b-a04a-2108f2feedf6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:47.181623 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22040baf-5a39-442b-a04a-2108f2feedf6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:47.214303 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a657184-3871-47a9-a79e-1157a9177197 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:47.226087 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90010b8e-c029-430f-a722-38be49b2ac3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:47.227172 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90010b8e-c029-430f-a722-38be49b2ac3e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:47.227662 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90010b8e-c029-430f-a722-38be49b2ac3e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:47.228142 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90010b8e-c029-430f-a722-38be49b2ac3e None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:47.229118 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90010b8e-c029-430f-a722-38be49b2ac3e None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:47.229627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90010b8e-c029-430f-a722-38be49b2ac3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:47.231274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90010b8e-c029-430f-a722-38be49b2ac3e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:47.241121 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 118/251] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:24:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:47.242284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90010b8e-c029-430f-a722-38be49b2ac3e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:47.272731 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336a0ac-4095-4f3b-be01-ce50d49a645f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:47.275482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336a0ac-4095-4f3b-be01-ce50d49a645f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:47.277306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336a0ac-4095-4f3b-be01-ce50d49a645f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:47.277794 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336a0ac-4095-4f3b-be01-ce50d49a645f None admin] PATH_INFO: `/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:47.280317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336a0ac-4095-4f3b-be01-ce50d49a645f None admin] RBAC: Authorizing `identity:get_project(project_id=bef8d3a846e34a8daf7eecb46a5fd2f8)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:47.282981 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336a0ac-4095-4f3b-be01-ce50d49a645f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:47.287381 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 134/252] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:46 2022] GET /identity/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8 => generated 302 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:47.290178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336a0ac-4095-4f3b-be01-ce50d49a645f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:47.336120 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:47.373606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:47.405664 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90010b8e-c029-430f-a722-38be49b2ac3e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:47.434027 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336a0ac-4095-4f3b-be01-ce50d49a645f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:47.444207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a615c-5ff0-4fa8-86b2-2cc695d0d5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:47.445297 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a615c-5ff0-4fa8-86b2-2cc695d0d5ef None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:47.445786 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a615c-5ff0-4fa8-86b2-2cc695d0d5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:47.446274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a615c-5ff0-4fa8-86b2-2cc695d0d5ef None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:47.447302 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a615c-5ff0-4fa8-86b2-2cc695d0d5ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:47.447721 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a615c-5ff0-4fa8-86b2-2cc695d0d5ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:47.449319 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a615c-5ff0-4fa8-86b2-2cc695d0d5ef None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:47.464419 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 119/253] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:47.465178 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a615c-5ff0-4fa8-86b2-2cc695d0d5ef None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:47.483710 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814211a0-73f0-4a39-a650-edbda77677d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:47.485005 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814211a0-73f0-4a39-a650-edbda77677d0 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:47.485390 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814211a0-73f0-4a39-a650-edbda77677d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:47.486011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814211a0-73f0-4a39-a650-edbda77677d0 None admin] PATH_INFO: `/v3/users/822895f606f24a74ad77852b46c49434` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:47.488115 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814211a0-73f0-4a39-a650-edbda77677d0 None admin] RBAC: Authorizing `identity:get_user(user_id=822895f606f24a74ad77852b46c49434)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:47.490309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814211a0-73f0-4a39-a650-edbda77677d0 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:47.492259 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 135/254] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:47 2022] GET /identity/v3/users/822895f606f24a74ad77852b46c49434 => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:47.493942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28505d09-9777-4978-86b2-0fe7640a47ac None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:47.494334 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28505d09-9777-4978-86b2-0fe7640a47ac None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:47.494745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28505d09-9777-4978-86b2-0fe7640a47ac None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:47.496129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 136/255] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:47.496924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28505d09-9777-4978-86b2-0fe7640a47ac None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:47.596730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a615c-5ff0-4fa8-86b2-2cc695d0d5ef None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:47.606405 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:47.614372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28505d09-9777-4978-86b2-0fe7640a47ac None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:47.634940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:47.651187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3236ad-8cbd-4dec-a1bd-e489d6015c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:47.652813 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3236ad-8cbd-4dec-a1bd-e489d6015c00 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:47.653651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3236ad-8cbd-4dec-a1bd-e489d6015c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:47.654143 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3236ad-8cbd-4dec-a1bd-e489d6015c00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:47.655955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3236ad-8cbd-4dec-a1bd-e489d6015c00 None admin] RBAC: Adding query filter params (role.id=8655041bde1948a1be442c6239496c22, scope.project.id=bef8d3a846e34a8daf7eecb46a5fd2f8, user.id=c41f416318cc4813877c7c3ea89bf8fd) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:47.656746 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3236ad-8cbd-4dec-a1bd-e489d6015c00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:47.659027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3236ad-8cbd-4dec-a1bd-e489d6015c00 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:47.660374 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87dd4cd-2d25-4028-bed9-0514c18da867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:47.661338 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87dd4cd-2d25-4028-bed9-0514c18da867 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:47.661609 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87dd4cd-2d25-4028-bed9-0514c18da867 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:47.661986 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87dd4cd-2d25-4028-bed9-0514c18da867 None admin] PATH_INFO: `/v3/users/6bb9868c931f4584aaa5206e6a324763` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:47.663492 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87dd4cd-2d25-4028-bed9-0514c18da867 None admin] RBAC: Authorizing `identity:get_user(user_id=6bb9868c931f4584aaa5206e6a324763)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:47.664772 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87dd4cd-2d25-4028-bed9-0514c18da867 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:47.666491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 137/256] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:47 2022] GET /identity/v3/users/6bb9868c931f4584aaa5206e6a324763 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:47.670745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87dd4cd-2d25-4028-bed9-0514c18da867 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:47.670945 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 120/257] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:24:47 2022] GET /identity/v3/role_assignments?user.id=c41f416318cc4813877c7c3ea89bf8fd&scope.project.id=bef8d3a846e34a8daf7eecb46a5fd2f8&role.id=8655041bde1948a1be442c6239496c22 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:47.672652 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34ab5c-39f7-4358-8282-0055631f3e43 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:47.672941 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34ab5c-39f7-4358-8282-0055631f3e43 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:47.673221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34ab5c-39f7-4358-8282-0055631f3e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:47.692234 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6c34ab5c-39f7-4358-8282-0055631f3e43 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:47.693508 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6c34ab5c-39f7-4358-8282-0055631f3e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:47.769379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 121/258] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:47.769987 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34ab5c-39f7-4358-8282-0055631f3e43 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:47.843846 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87dd4cd-2d25-4028-bed9-0514c18da867 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:47.901523 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df56e62-2136-4046-b375-9cabc6e03489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:47.902706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df56e62-2136-4046-b375-9cabc6e03489 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:47.902996 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df56e62-2136-4046-b375-9cabc6e03489 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:47.903268 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df56e62-2136-4046-b375-9cabc6e03489 None admin] PATH_INFO: `/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:47.905165 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df56e62-2136-4046-b375-9cabc6e03489 None admin] RBAC: Authorizing `identity:get_project(project_id=bef8d3a846e34a8daf7eecb46a5fd2f8)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:47.906638 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df56e62-2136-4046-b375-9cabc6e03489 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:47.909710 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 138/259] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:47 2022] GET /identity/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:47.910592 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df56e62-2136-4046-b375-9cabc6e03489 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:47.963801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:47.968242 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34ab5c-39f7-4358-8282-0055631f3e43 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:47.980232 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:48.020110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24bdbfe-728d-4f8d-a162-c25148195023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:48.020696 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bdbfe-728d-4f8d-a162-c25148195023 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:48.020984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bdbfe-728d-4f8d-a162-c25148195023 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:48.021262 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bdbfe-728d-4f8d-a162-c25148195023 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:48.066673 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df56e62-2136-4046-b375-9cabc6e03489 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:48.098854 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24bdbfe-728d-4f8d-a162-c25148195023 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:48.100450 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24bdbfe-728d-4f8d-a162-c25148195023 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:48.163351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5961b-a756-433d-bf68-f4ba45556bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:48.164182 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5961b-a756-433d-bf68-f4ba45556bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:48.164467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5961b-a756-433d-bf68-f4ba45556bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:48.164743 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5961b-a756-433d-bf68-f4ba45556bb0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:48.191578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5961b-a756-433d-bf68-f4ba45556bb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:48.207732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-65c5961b-a756-433d-bf68-f4ba45556bb0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:24:48.207732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:48.207732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:48.207732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:48.207732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:48.207732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:48.207732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:48.207732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:48.207732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:48.207732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:48.207732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:48.207732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:48.207732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:48.207732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:48.207732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:48.207732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:48.208726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:48.208726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:48.208726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:48.208726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:48.208726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:48.208726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:48.208726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:48.208726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:48.208726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:48.208726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:48.208726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:48.208726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:24:48.208726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:48.208726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:48.208726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:48.208726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:48.209532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:48.209532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:48.209532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:48.209532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:48.209532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:48.209532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:48.209532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:48.209532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:48.209532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:48.209532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:48.209532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:24:48.209532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:24:48.209532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 139/260] 10.222.0.67 () {56 vars in 1114 bytes} [Tue May 31 14:24:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:48.210872 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5961b-a756-433d-bf68-f4ba45556bb0 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:48.213589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:48.219517 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 122/261] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:24:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:24:48.224299 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24bdbfe-728d-4f8d-a162-c25148195023 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:48.422688 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24bdbfe-728d-4f8d-a162-c25148195023 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:48.423315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5961b-a756-433d-bf68-f4ba45556bb0 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:48.433714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:48.461780 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548727ae-705c-4717-8800-802654a2db32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:48.462225 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548727ae-705c-4717-8800-802654a2db32 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:48.462521 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548727ae-705c-4717-8800-802654a2db32 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:48.462850 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548727ae-705c-4717-8800-802654a2db32 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:48.463615 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548727ae-705c-4717-8800-802654a2db32 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:48.463925 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548727ae-705c-4717-8800-802654a2db32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:48.465333 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548727ae-705c-4717-8800-802654a2db32 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:48.474832 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 123/262] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:24:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:48.475738 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548727ae-705c-4717-8800-802654a2db32 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:48.520277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc66e8-e858-4904-ae6b-505d10b3ffa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:48.521183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc66e8-e858-4904-ae6b-505d10b3ffa2 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:48.521463 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc66e8-e858-4904-ae6b-505d10b3ffa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:48.521736 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc66e8-e858-4904-ae6b-505d10b3ffa2 None admin] PATH_INFO: `/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8/users/822895f606f24a74ad77852b46c49434/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:48.527440 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc66e8-e858-4904-ae6b-505d10b3ffa2 None admin] RBAC: Authorizing `identity:create_grant(project_id=bef8d3a846e34a8daf7eecb46a5fd2f8, user_id=822895f606f24a74ad77852b46c49434, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:48.529517 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc66e8-e858-4904-ae6b-505d10b3ffa2 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:48.541624 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:48.573959 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 140/263] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:24:48 2022] PUT /identity/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8/users/822895f606f24a74ad77852b46c49434/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:24:48.574921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc66e8-e858-4904-ae6b-505d10b3ffa2 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:48.584985 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548727ae-705c-4717-8800-802654a2db32 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:48.626777 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f3938-8604-4405-941f-f6418873adc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:48.627762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f3938-8604-4405-941f-f6418873adc2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:48.628050 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f3938-8604-4405-941f-f6418873adc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:48.628335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f3938-8604-4405-941f-f6418873adc2 None admin] PATH_INFO: `/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:48.629899 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f3938-8604-4405-941f-f6418873adc2 None admin] RBAC: Authorizing `identity:get_project(project_id=bef8d3a846e34a8daf7eecb46a5fd2f8)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:48.631476 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f3938-8604-4405-941f-f6418873adc2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:48.633494 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 124/264] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:48 2022] GET /identity/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:48.640749 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f3938-8604-4405-941f-f6418873adc2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:48.754776 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f3938-8604-4405-941f-f6418873adc2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:48.791893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38312fb5-d800-4d9f-90ca-d3a8e75055fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:48.792569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38312fb5-d800-4d9f-90ca-d3a8e75055fb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:48.792861 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38312fb5-d800-4d9f-90ca-d3a8e75055fb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:48.793145 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38312fb5-d800-4d9f-90ca-d3a8e75055fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:48.794157 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38312fb5-d800-4d9f-90ca-d3a8e75055fb None admin] RBAC: Adding query filter params (role.id=e8d679596ffb45939229d1e2a13ce9b0, scope.project.id=bef8d3a846e34a8daf7eecb46a5fd2f8, user.id=6bb9868c931f4584aaa5206e6a324763) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:48.794479 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38312fb5-d800-4d9f-90ca-d3a8e75055fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:48.796009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38312fb5-d800-4d9f-90ca-d3a8e75055fb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:48.806718 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 125/265] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:24:48 2022] GET /identity/v3/role_assignments?user.id=6bb9868c931f4584aaa5206e6a324763&scope.project.id=bef8d3a846e34a8daf7eecb46a5fd2f8&role.id=e8d679596ffb45939229d1e2a13ce9b0 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:48.807063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:48.859486 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc66e8-e858-4904-ae6b-505d10b3ffa2 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:48.925088 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc95e1-4375-4b4d-a652-0f9cda6b9eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:48.926392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc95e1-4375-4b4d-a652-0f9cda6b9eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:48.926944 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc95e1-4375-4b4d-a652-0f9cda6b9eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:48.927422 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc95e1-4375-4b4d-a652-0f9cda6b9eb9 None admin] PATH_INFO: `/v3/users/6bb9868c931f4584aaa5206e6a324763` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:48.928923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc95e1-4375-4b4d-a652-0f9cda6b9eb9 None admin] RBAC: Authorizing `identity:get_user(user_id=6bb9868c931f4584aaa5206e6a324763)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:48.930610 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc95e1-4375-4b4d-a652-0f9cda6b9eb9 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:48.932608 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 141/266] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:48 2022] GET /identity/v3/users/6bb9868c931f4584aaa5206e6a324763 => generated 283 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:48.939157 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38312fb5-d800-4d9f-90ca-d3a8e75055fb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:49.085157 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38312fb5-d800-4d9f-90ca-d3a8e75055fb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:49.143498 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2aa17-e3a6-4686-b2af-b8c1f016ed63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:49.144453 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2aa17-e3a6-4686-b2af-b8c1f016ed63 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:49.144967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2aa17-e3a6-4686-b2af-b8c1f016ed63 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:49.145448 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2aa17-e3a6-4686-b2af-b8c1f016ed63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:49.168834 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121cb761-379c-4222-8f32-588dc838a437 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:49.169982 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121cb761-379c-4222-8f32-588dc838a437 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:49.170475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121cb761-379c-4222-8f32-588dc838a437 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:49.172124 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 142/267] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:49.177427 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffc4e9-917c-4ab6-afb3-3fb052f8cd72 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:49.177718 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffc4e9-917c-4ab6-afb3-3fb052f8cd72 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:49.177987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffc4e9-917c-4ab6-afb3-3fb052f8cd72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:49.199730 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-18ffc4e9-917c-4ab6-afb3-3fb052f8cd72 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:49.200972 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-18ffc4e9-917c-4ab6-afb3-3fb052f8cd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:49.261228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:49.279107 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:49.291552 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c2aa17-e3a6-4686-b2af-b8c1f016ed63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:49.293070 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c2aa17-e3a6-4686-b2af-b8c1f016ed63 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:49.334101 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 143/268] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 31 14:24:49.335547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5e6e9-18f2-4e91-a6b3-c41cca4974d8 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:49.335783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5e6e9-18f2-4e91-a6b3-c41cca4974d8 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:49.336060 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5e6e9-18f2-4e91-a6b3-c41cca4974d8 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:49.337334 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 144/269] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:49.346989 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd0d56-f7a8-4169-a038-74fdaa10e63b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:49.347276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd0d56-f7a8-4169-a038-74fdaa10e63b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:49.347554 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd0d56-f7a8-4169-a038-74fdaa10e63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:49.376080 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-cffd0d56-f7a8-4169-a038-74fdaa10e63b None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:49.377274 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cffd0d56-f7a8-4169-a038-74fdaa10e63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:49.433451 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 126/270] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:24:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:24:49.434305 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2aa17-e3a6-4686-b2af-b8c1f016ed63 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:49.460012 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 145/271] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 31 14:24:49.461055 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd0d56-f7a8-4169-a038-74fdaa10e63b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:49.560540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:49.562866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:49.568645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd0d56-f7a8-4169-a038-74fdaa10e63b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:49.578006 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2aa17-e3a6-4686-b2af-b8c1f016ed63 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:49.604091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790eb16-e81c-42ef-8268-1ed6b46e1aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:49.604973 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790eb16-e81c-42ef-8268-1ed6b46e1aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:49.605224 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790eb16-e81c-42ef-8268-1ed6b46e1aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:49.605495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790eb16-e81c-42ef-8268-1ed6b46e1aa4 None admin] PATH_INFO: `/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:49.607233 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b790eb16-e81c-42ef-8268-1ed6b46e1aa4 None admin] RBAC: Authorizing `identity:get_project(project_id=bef8d3a846e34a8daf7eecb46a5fd2f8)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:49.608680 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b790eb16-e81c-42ef-8268-1ed6b46e1aa4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:49.614783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 146/272] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:49 2022] GET /identity/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:49.614783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790eb16-e81c-42ef-8268-1ed6b46e1aa4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:49.628001 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf01092-0685-49e7-bd1c-e9fc806682be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:49.629146 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf01092-0685-49e7-bd1c-e9fc806682be None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:49.629788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf01092-0685-49e7-bd1c-e9fc806682be None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:49.630262 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf01092-0685-49e7-bd1c-e9fc806682be None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:49.642202 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf01092-0685-49e7-bd1c-e9fc806682be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:49.650537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-3cf01092-0685-49e7-bd1c-e9fc806682be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 31 14:24:49.650537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:49.650537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:49.650537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:49.650537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:49.650537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:49.650537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:49.650537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:49.650537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:49.650537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:49.650537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:49.650537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:49.650537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:49.650537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:49.650537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:49.650537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:49.651437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:49.651437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:49.651437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:49.651437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:49.651437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:49.651437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:49.651437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:49.651437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:49.651437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:49.651437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:49.651437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:49.651437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:24:49.651437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:49.651437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:49.651437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:49.651437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:49.652230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:49.652230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:49.652230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:49.652230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:49.652230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:49.652230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:49.652230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:49.652230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:49.652230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:49.652230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:49.652230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 31 14:24:49.652230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:24:49.655675 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 127/273] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:49.656715 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf01092-0685-49e7-bd1c-e9fc806682be None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:49.717894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790eb16-e81c-42ef-8268-1ed6b46e1aa4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:49.752052 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e2cb9-d09a-4131-9583-721fbbf455ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:49.752862 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e2cb9-d09a-4131-9583-721fbbf455ce None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:49.753404 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e2cb9-d09a-4131-9583-721fbbf455ce None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:49.753404 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e2cb9-d09a-4131-9583-721fbbf455ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:49.754662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e2cb9-d09a-4131-9583-721fbbf455ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:49.756085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e2cb9-d09a-4131-9583-721fbbf455ce None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:49.757879 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 147/274] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:24:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:49.758468 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e2cb9-d09a-4131-9583-721fbbf455ce None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:49.775377 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf01092-0685-49e7-bd1c-e9fc806682be None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:49.793451 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:49.815260 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede506a4-312d-41da-8950-b238752b0ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:49.816375 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede506a4-312d-41da-8950-b238752b0ccf None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:49.816884 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede506a4-312d-41da-8950-b238752b0ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:49.817375 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede506a4-312d-41da-8950-b238752b0ccf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:49.818504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede506a4-312d-41da-8950-b238752b0ccf None admin] RBAC: Adding query filter params (role.id=1b9c18d4b8b540bf9825080748cec3bc, scope.project.id=bef8d3a846e34a8daf7eecb46a5fd2f8, user.id=6bb9868c931f4584aaa5206e6a324763) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:49.819135 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede506a4-312d-41da-8950-b238752b0ccf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:49.820790 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede506a4-312d-41da-8950-b238752b0ccf None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:49.828902 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 128/275] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:24:49 2022] GET /identity/v3/role_assignments?user.id=6bb9868c931f4584aaa5206e6a324763&scope.project.id=bef8d3a846e34a8daf7eecb46a5fd2f8&role.id=1b9c18d4b8b540bf9825080748cec3bc => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:49.835380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede506a4-312d-41da-8950-b238752b0ccf None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:49.862975 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e2cb9-d09a-4131-9583-721fbbf455ce None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:49.872556 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:49.912804 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c86da-bfca-42e9-ba08-8aea39fa4e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:49.913496 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c86da-bfca-42e9-ba08-8aea39fa4e53 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:49.913777 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c86da-bfca-42e9-ba08-8aea39fa4e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:49.914048 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c86da-bfca-42e9-ba08-8aea39fa4e53 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:49.914833 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4c86da-bfca-42e9-ba08-8aea39fa4e53 None admin] RBAC: Adding query filter params (name=member) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:49.915137 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4c86da-bfca-42e9-ba08-8aea39fa4e53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:49.916524 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4c86da-bfca-42e9-ba08-8aea39fa4e53 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:49.936074 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 148/276] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:24:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:49.959645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c86da-bfca-42e9-ba08-8aea39fa4e53 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:50.040781 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede506a4-312d-41da-8950-b238752b0ccf None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:50.080490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579e0f4-d907-4098-9000-b7b395180c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:50.081195 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579e0f4-d907-4098-9000-b7b395180c75 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:50.081476 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579e0f4-d907-4098-9000-b7b395180c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:50.081760 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579e0f4-d907-4098-9000-b7b395180c75 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:50.082483 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6579e0f4-d907-4098-9000-b7b395180c75 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:50.086968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6579e0f4-d907-4098-9000-b7b395180c75 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:50.110485 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 129/277] 10.222.0.67 () {60 vars in 1132 bytes} [Tue May 31 14:24:49 2022] POST /identity/v3/users => generated 293 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:24:50.147165 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:50.151462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c86da-bfca-42e9-ba08-8aea39fa4e53 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:50.194016 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cbd2b4-c011-42d9-ac01-10d7f40023f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:50.194964 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbd2b4-c011-42d9-ac01-10d7f40023f2 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:50.195301 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbd2b4-c011-42d9-ac01-10d7f40023f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:50.195582 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbd2b4-c011-42d9-ac01-10d7f40023f2 None admin] PATH_INFO: `/v3/users/7411ede0d7d749b5aa52558188b3a3d9` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:50.212533 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cbd2b4-c011-42d9-ac01-10d7f40023f2 None admin] RBAC: Authorizing `identity:get_user(user_id=7411ede0d7d749b5aa52558188b3a3d9)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:50.214088 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cbd2b4-c011-42d9-ac01-10d7f40023f2 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:50.216390 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 149/278] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:49 2022] GET /identity/v3/users/7411ede0d7d749b5aa52558188b3a3d9 => generated 293 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:24:50.222042 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579e0f4-d907-4098-9000-b7b395180c75 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:50.314299 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:50.350336 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579e0f4-d907-4098-9000-b7b395180c75 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:50.392627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c286184-5752-4b20-8f74-23843e065e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:50.393216 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c286184-5752-4b20-8f74-23843e065e58 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:50.393510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c286184-5752-4b20-8f74-23843e065e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:50.393797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c286184-5752-4b20-8f74-23843e065e58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:50.394682 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c286184-5752-4b20-8f74-23843e065e58 None admin] RBAC: Adding query filter params (role.id=9ce8fdecf52d482d894fdeec4f530624, user.id=7411ede0d7d749b5aa52558188b3a3d9, scope.system=all) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:50.394993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c286184-5752-4b20-8f74-23843e065e58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:50.396448 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c286184-5752-4b20-8f74-23843e065e58 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:50.404788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 130/279] 10.222.0.67 () {56 vars in 1308 bytes} [Tue May 31 14:24:50 2022] GET /identity/v3/role_assignments?user.id=7411ede0d7d749b5aa52558188b3a3d9&scope.system=all&role.id=9ce8fdecf52d482d894fdeec4f530624 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:50.573769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc3a51-7450-4483-9aa9-14c63a0171c8 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:50.574755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc3a51-7450-4483-9aa9-14c63a0171c8 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:50.575447 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc3a51-7450-4483-9aa9-14c63a0171c8 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:50.577596 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 150/280] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:50.583405 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24279594-8689-4b3a-ad7e-154f598018a6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:50.583900 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24279594-8689-4b3a-ad7e-154f598018a6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:50.583975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24279594-8689-4b3a-ad7e-154f598018a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:50.603035 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-24279594-8689-4b3a-ad7e-154f598018a6 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:50.604268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-24279594-8689-4b3a-ad7e-154f598018a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:50.679404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 131/281] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:50.720051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc3a51-7450-4483-9aa9-14c63a0171c8 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:50.872354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc3a51-7450-4483-9aa9-14c63a0171c8 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:50.922803 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54391c22-3725-443f-b646-6f0a81c6f40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:50.923846 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54391c22-3725-443f-b646-6f0a81c6f40e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:50.924302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54391c22-3725-443f-b646-6f0a81c6f40e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:50.924781 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54391c22-3725-443f-b646-6f0a81c6f40e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:50.940262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54391c22-3725-443f-b646-6f0a81c6f40e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:50.948252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-54391c22-3725-443f-b646-6f0a81c6f40e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 31 14:24:50.948252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:50.948252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:50.948252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:50.948252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:50.948252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:50.948252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:50.948252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:50.948252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:50.948252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:50.948252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:50.948252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:50.948252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:50.948252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:50.948252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:50.948252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:50.949330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:50.949330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:50.949330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:50.949330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:50.949330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:50.949330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:50.949330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:50.949330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:50.949330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:50.949330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:50.949330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:50.949330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:24:50.949330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:50.949330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:50.949330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:50.949330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:50.950360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:50.950360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:50.950360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:50.950360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:50.950360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:50.950360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:50.950360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:50.950360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:50.950360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:50.950360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:50.950360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 31 14:24:50.950360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:24:50.952928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 151/282] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:50.958371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24279594-8689-4b3a-ad7e-154f598018a6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:51.057259 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:51.097029 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24279594-8689-4b3a-ad7e-154f598018a6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:51.152430 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23673d0e-ab40-4cf2-954d-8c1cf620b803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:51.153702 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23673d0e-ab40-4cf2-954d-8c1cf620b803 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:51.154033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23673d0e-ab40-4cf2-954d-8c1cf620b803 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:51.154319 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23673d0e-ab40-4cf2-954d-8c1cf620b803 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:51.155121 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23673d0e-ab40-4cf2-954d-8c1cf620b803 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:51.155434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23673d0e-ab40-4cf2-954d-8c1cf620b803 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:51.156827 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23673d0e-ab40-4cf2-954d-8c1cf620b803 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:51.171292 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 132/283] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:24:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:51.176472 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54391c22-3725-443f-b646-6f0a81c6f40e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:51.201501 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:51.314245 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54391c22-3725-443f-b646-6f0a81c6f40e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:51.358788 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d4711-50b7-464c-8e7a-1dc2bd6bb64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:51.359687 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d4711-50b7-464c-8e7a-1dc2bd6bb64d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:51.359971 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d4711-50b7-464c-8e7a-1dc2bd6bb64d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:51.360250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d4711-50b7-464c-8e7a-1dc2bd6bb64d None admin] PATH_INFO: `/v3/users/c41f416318cc4813877c7c3ea89bf8fd` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:51.361500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d4711-50b7-464c-8e7a-1dc2bd6bb64d None admin] RBAC: Authorizing `identity:get_user(user_id=c41f416318cc4813877c7c3ea89bf8fd)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:51.362996 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d4711-50b7-464c-8e7a-1dc2bd6bb64d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:51.364642 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 152/284] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:51 2022] GET /identity/v3/users/c41f416318cc4813877c7c3ea89bf8fd => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:51.372217 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23673d0e-ab40-4cf2-954d-8c1cf620b803 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:51.504797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23673d0e-ab40-4cf2-954d-8c1cf620b803 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:51.551952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:51.560262 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea0c02-045f-4e72-8d25-a6666de151df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:51.561713 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea0c02-045f-4e72-8d25-a6666de151df None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:51.562417 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea0c02-045f-4e72-8d25-a6666de151df None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:51.563141 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea0c02-045f-4e72-8d25-a6666de151df None admin] PATH_INFO: `/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:51.564376 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dea0c02-045f-4e72-8d25-a6666de151df None admin] RBAC: Authorizing `identity:get_project(project_id=bef8d3a846e34a8daf7eecb46a5fd2f8)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:51.565862 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dea0c02-045f-4e72-8d25-a6666de151df None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:51.567981 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 133/285] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:51 2022] GET /identity/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:51.573388 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d4711-50b7-464c-8e7a-1dc2bd6bb64d None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:51.662802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc6a5a-55f3-443d-8da8-31779894f459 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:51.663315 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc6a5a-55f3-443d-8da8-31779894f459 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:51.663590 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc6a5a-55f3-443d-8da8-31779894f459 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:51.666747 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 134/286] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:51.676157 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd647c-8839-4eca-9e74-e0b0939d24e7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:51.676293 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd647c-8839-4eca-9e74-e0b0939d24e7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:51.676663 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd647c-8839-4eca-9e74-e0b0939d24e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:51.708387 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-40dd647c-8839-4eca-9e74-e0b0939d24e7 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:51.709568 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-40dd647c-8839-4eca-9e74-e0b0939d24e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:51.712831 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d4711-50b7-464c-8e7a-1dc2bd6bb64d None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:51.725510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:51.790238 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f3845-6610-42d4-a88f-f0452db4678c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:51.791257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f3845-6610-42d4-a88f-f0452db4678c None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:51.791549 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f3845-6610-42d4-a88f-f0452db4678c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:51.791843 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f3845-6610-42d4-a88f-f0452db4678c None admin] PATH_INFO: `/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8/users/c41f416318cc4813877c7c3ea89bf8fd/roles/8655041bde1948a1be442c6239496c22` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:51.800429 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f3845-6610-42d4-a88f-f0452db4678c None admin] RBAC: Authorizing `identity:create_grant(project_id=bef8d3a846e34a8daf7eecb46a5fd2f8, user_id=c41f416318cc4813877c7c3ea89bf8fd, role_id=8655041bde1948a1be442c6239496c22)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:51.802060 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f3845-6610-42d4-a88f-f0452db4678c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:51.803038 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 135/287] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:51.827617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd647c-8839-4eca-9e74-e0b0939d24e7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:51.837899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 153/288] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:24:51 2022] PUT /identity/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8/users/c41f416318cc4813877c7c3ea89bf8fd/roles/8655041bde1948a1be442c6239496c22 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:24:51.979089 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd647c-8839-4eca-9e74-e0b0939d24e7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:52.020292 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:52.032191 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bccaad-0370-4492-be1d-b54513223378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:52.033037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bccaad-0370-4492-be1d-b54513223378 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:52.033326 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bccaad-0370-4492-be1d-b54513223378 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:52.033604 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bccaad-0370-4492-be1d-b54513223378 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:52.051190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bccaad-0370-4492-be1d-b54513223378 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:52.059286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-d7bccaad-0370-4492-be1d-b54513223378 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 31 14:24:52.059286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:52.059286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:52.059286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:52.059286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:52.059286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:52.059286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:52.059286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:52.059286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:52.059286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:52.059286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:52.059286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:52.059286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:52.059286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:52.059286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:52.059286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:52.060291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:52.060291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:52.060291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:52.060291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:52.060291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:52.060291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:52.060291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:52.060291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:52.060291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:52.060291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:52.060291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:52.060291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:24:52.060291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:52.060291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:52.060291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:52.060291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:52.061093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:52.061093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:52.061093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:52.061093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:52.061093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:52.061093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:52.061093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:52.061093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:52.061093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:52.061093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:52.061093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 31 14:24:52.061093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:24:52.061093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 136/289] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:24:52.067714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f3845-6610-42d4-a88f-f0452db4678c None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:52.176876 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f3845-6610-42d4-a88f-f0452db4678c None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:52.212983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0b732-9829-44d6-b601-4ff6cb0ca652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:52.213671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0b732-9829-44d6-b601-4ff6cb0ca652 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:52.213960 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0b732-9829-44d6-b601-4ff6cb0ca652 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:52.214238 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0b732-9829-44d6-b601-4ff6cb0ca652 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:52.215022 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b0b732-9829-44d6-b601-4ff6cb0ca652 None admin] RBAC: Adding query filter params (name=member) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:52.215338 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b0b732-9829-44d6-b601-4ff6cb0ca652 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:52.217317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b0b732-9829-44d6-b601-4ff6cb0ca652 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:52.226521 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 154/290] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:24:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:52.231844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bccaad-0370-4492-be1d-b54513223378 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:52.397529 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bccaad-0370-4492-be1d-b54513223378 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:52.442885 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363760c-03f1-4546-a5fa-988467a4a1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:52.444176 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363760c-03f1-4546-a5fa-988467a4a1bd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:52.444663 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363760c-03f1-4546-a5fa-988467a4a1bd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:52.445139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363760c-03f1-4546-a5fa-988467a4a1bd None admin] PATH_INFO: `/v3/users/822895f606f24a74ad77852b46c49434` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:52.451416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8363760c-03f1-4546-a5fa-988467a4a1bd None admin] RBAC: Authorizing `identity:get_user(user_id=822895f606f24a74ad77852b46c49434)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:52.453380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8363760c-03f1-4546-a5fa-988467a4a1bd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:52.455289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 137/291] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:52 2022] GET /identity/v3/users/822895f606f24a74ad77852b46c49434 => generated 297 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:52.461209 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0b732-9829-44d6-b601-4ff6cb0ca652 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:52.469881 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:52.609071 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0b732-9829-44d6-b601-4ff6cb0ca652 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:52.664394 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc31ef-63b0-42f8-9fa6-a0f2bb35ba86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:52.664989 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc31ef-63b0-42f8-9fa6-a0f2bb35ba86 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:52.665278 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc31ef-63b0-42f8-9fa6-a0f2bb35ba86 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:52.665558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc31ef-63b0-42f8-9fa6-a0f2bb35ba86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:52.774324 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc31ef-63b0-42f8-9fa6-a0f2bb35ba86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:52.775873 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc31ef-63b0-42f8-9fa6-a0f2bb35ba86 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:52.783704 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:52.934900 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 155/292] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:24:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:24:52.940640 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363760c-03f1-4546-a5fa-988467a4a1bd None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:53.026179 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:53.063891 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b57d2d-2540-4d88-8b8a-58a2b63648fc None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:53.064187 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b57d2d-2540-4d88-8b8a-58a2b63648fc None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:53.064472 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b57d2d-2540-4d88-8b8a-58a2b63648fc None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:53.070021 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 156/293] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:53.071674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3072f9-d677-412a-bcd9-71999726c05b None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:53.071969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3072f9-d677-412a-bcd9-71999726c05b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:53.072255 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3072f9-d677-412a-bcd9-71999726c05b None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:53.073556 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 157/294] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 31 14:24:53.079739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d468892-4aad-402e-af4a-7c7ea3429eed None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:53.080034 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d468892-4aad-402e-af4a-7c7ea3429eed None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:53.080315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d468892-4aad-402e-af4a-7c7ea3429eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:53.092875 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363760c-03f1-4546-a5fa-988467a4a1bd None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:53.098190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-4d468892-4aad-402e-af4a-7c7ea3429eed None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:53.099437 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4d468892-4aad-402e-af4a-7c7ea3429eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:53.137146 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525a379-a7f8-419d-b25d-5424e0293851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:53.139103 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525a379-a7f8-419d-b25d-5424e0293851 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:53.139827 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525a379-a7f8-419d-b25d-5424e0293851 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:53.141028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525a379-a7f8-419d-b25d-5424e0293851 None admin] PATH_INFO: `/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:53.142897 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6525a379-a7f8-419d-b25d-5424e0293851 None admin] RBAC: Authorizing `identity:get_project(project_id=bef8d3a846e34a8daf7eecb46a5fd2f8)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:53.144608 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6525a379-a7f8-419d-b25d-5424e0293851 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:53.148039 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 138/295] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:52 2022] GET /identity/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:53.154594 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f119c-c5f0-4358-9a7e-60bf36502014 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:53.155110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f119c-c5f0-4358-9a7e-60bf36502014 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:53.156064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f119c-c5f0-4358-9a7e-60bf36502014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:53.179078 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1f7f119c-c5f0-4358-9a7e-60bf36502014 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:53.180730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1f7f119c-c5f0-4358-9a7e-60bf36502014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:53.183085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 158/296] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 31 14:24:53.191754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d468892-4aad-402e-af4a-7c7ea3429eed None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:53.249954 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:53.272593 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 139/297] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:53.273680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f119c-c5f0-4358-9a7e-60bf36502014 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:53.327931 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d468892-4aad-402e-af4a-7c7ea3429eed None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:53.335517 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:53.363529 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a214a-711a-42a0-b56d-fc1e7d0ad9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:53.364227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a214a-711a-42a0-b56d-fc1e7d0ad9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:53.364519 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a214a-711a-42a0-b56d-fc1e7d0ad9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:53.364801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a214a-711a-42a0-b56d-fc1e7d0ad9b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:53.365638 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a214a-711a-42a0-b56d-fc1e7d0ad9b1 None admin] RBAC: Adding query filter params (role.id=9ce8fdecf52d482d894fdeec4f530624, scope.project.id=bef8d3a846e34a8daf7eecb46a5fd2f8, user.id=822895f606f24a74ad77852b46c49434) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:53.365957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a214a-711a-42a0-b56d-fc1e7d0ad9b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:53.367428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a214a-711a-42a0-b56d-fc1e7d0ad9b1 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:53.374534 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 159/298] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:24:53 2022] GET /identity/v3/role_assignments?user.id=822895f606f24a74ad77852b46c49434&scope.project.id=bef8d3a846e34a8daf7eecb46a5fd2f8&role.id=9ce8fdecf52d482d894fdeec4f530624 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:53.375602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a214a-711a-42a0-b56d-fc1e7d0ad9b1 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:53.383517 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f119c-c5f0-4358-9a7e-60bf36502014 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:53.444749 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a0972-5a4e-4b93-b549-ae2de354ec7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:53.445500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a0972-5a4e-4b93-b549-ae2de354ec7c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:53.445788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a0972-5a4e-4b93-b549-ae2de354ec7c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:53.446066 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a0972-5a4e-4b93-b549-ae2de354ec7c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:53.469918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a0972-5a4e-4b93-b549-ae2de354ec7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:53.491745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-693a0972-5a4e-4b93-b549-ae2de354ec7c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 31 14:24:53.491745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:53.491745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:53.491745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:53.491745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:53.491745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:53.491745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:53.491745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:53.491745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:53.491745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:53.491745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:53.491745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:53.491745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:53.491745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:53.491745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:53.491745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:53.492695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:53.492695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:53.492695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:53.492695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:53.492695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:53.492695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:53.492695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:53.492695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:53.492695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:53.492695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:53.492695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:53.492695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:24:53.492695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:53.492695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:53.492695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:53.492695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:53.493505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:53.493505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:53.493505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:53.493505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:53.493505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:53.493505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:53.493505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:53.493505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:53.493505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:53.493505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:53.493505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 31 14:24:53.493505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:24:53.493505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 140/299] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:53.499911 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:53.504603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a214a-711a-42a0-b56d-fc1e7d0ad9b1 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:53.520185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a0972-5a4e-4b93-b549-ae2de354ec7c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:53.544928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffdd6f-a217-41cd-b50b-6e8d7dc26c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:53.545758 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffdd6f-a217-41cd-b50b-6e8d7dc26c07 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:53.546046 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffdd6f-a217-41cd-b50b-6e8d7dc26c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:53.546329 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffdd6f-a217-41cd-b50b-6e8d7dc26c07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:53.547628 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecffdd6f-a217-41cd-b50b-6e8d7dc26c07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:53.549116 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecffdd6f-a217-41cd-b50b-6e8d7dc26c07 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:53.550938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 160/300] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:24:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:53.556811 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffdd6f-a217-41cd-b50b-6e8d7dc26c07 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:53.667322 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffdd6f-a217-41cd-b50b-6e8d7dc26c07 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:53.690874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a0972-5a4e-4b93-b549-ae2de354ec7c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:53.709705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:53.717882 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf368c49-ba68-4d5f-930c-554f28e9dd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:53.718664 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf368c49-ba68-4d5f-930c-554f28e9dd08 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:53.719062 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf368c49-ba68-4d5f-930c-554f28e9dd08 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:53.719486 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf368c49-ba68-4d5f-930c-554f28e9dd08 None admin] PATH_INFO: `/v3/groups` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:53.720561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf368c49-ba68-4d5f-930c-554f28e9dd08 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:53.722053 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf368c49-ba68-4d5f-930c-554f28e9dd08 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:53.737768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20de287-e104-44e8-b668-d4035eca3477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:53.738469 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20de287-e104-44e8-b668-d4035eca3477 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:53.738784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20de287-e104-44e8-b668-d4035eca3477 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:53.739065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20de287-e104-44e8-b668-d4035eca3477 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:53.739225 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 161/301] 10.222.0.67 () {60 vars in 1134 bytes} [Tue May 31 14:24:53 2022] POST /identity/v3/groups => generated 229 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:24:53.740493 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f75d9-a4b6-4cdd-af0d-853e741bf3c0 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:53.740775 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f75d9-a4b6-4cdd-af0d-853e741bf3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:53.741046 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f75d9-a4b6-4cdd-af0d-853e741bf3c0 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:53.742782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 162/302] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:53.743581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20de287-e104-44e8-b668-d4035eca3477 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:53.744022 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20de287-e104-44e8-b668-d4035eca3477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:53.745449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20de287-e104-44e8-b668-d4035eca3477 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:53.752688 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2804812-4d4b-4b7c-811c-1fe134b1ef75 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:53.753409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2804812-4d4b-4b7c-811c-1fe134b1ef75 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:53.753692 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2804812-4d4b-4b7c-811c-1fe134b1ef75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:53.763619 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 141/303] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:24:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:53.770706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20de287-e104-44e8-b668-d4035eca3477 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:53.783584 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-b2804812-4d4b-4b7c-811c-1fe134b1ef75 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:53.785503 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b2804812-4d4b-4b7c-811c-1fe134b1ef75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:53.814499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:53.883597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 163/304] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 31 14:24:53.908142 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20de287-e104-44e8-b668-d4035eca3477 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:53.912062 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2804812-4d4b-4b7c-811c-1fe134b1ef75 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:53.971430 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235820c-d77f-4711-97cd-73632ffbf5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:53.972362 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235820c-d77f-4711-97cd-73632ffbf5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:53.972656 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235820c-d77f-4711-97cd-73632ffbf5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:53.972934 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235820c-d77f-4711-97cd-73632ffbf5a6 None admin] PATH_INFO: `/v3/users/89cad0e7021f447dbb7268c90cfbcdb9` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:53.989702 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a235820c-d77f-4711-97cd-73632ffbf5a6 None admin] RBAC: Authorizing `identity:get_user(user_id=89cad0e7021f447dbb7268c90cfbcdb9)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:53.991623 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a235820c-d77f-4711-97cd-73632ffbf5a6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:53.993541 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 142/305] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:53 2022] GET /identity/v3/users/89cad0e7021f447dbb7268c90cfbcdb9 => generated 293 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:24:53.999040 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235820c-d77f-4711-97cd-73632ffbf5a6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:54.037551 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2804812-4d4b-4b7c-811c-1fe134b1ef75 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:54.042383 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:54.083819 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e7e8d-c97a-4fed-9297-72b6e79c0a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:54.084631 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e7e8d-c97a-4fed-9297-72b6e79c0a44 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:54.084917 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e7e8d-c97a-4fed-9297-72b6e79c0a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:54.085198 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e7e8d-c97a-4fed-9297-72b6e79c0a44 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:54.096436 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e7e8d-c97a-4fed-9297-72b6e79c0a44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:54.104256 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-1e3e7e8d-c97a-4fed-9297-72b6e79c0a44 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 31 14:24:54.104256 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:54.104256 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:54.104256 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:54.104256 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:54.104256 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:54.104256 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:54.104256 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:54.104256 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:54.104256 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:54.104256 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:54.104256 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:54.104256 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:54.104256 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:54.104256 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:54.104256 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:54.105155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:54.105155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:54.105155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:54.105155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:54.105155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:54.105155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:54.105155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:54.105155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:54.105155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:54.105155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:54.105155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:54.105155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:24:54.105155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:54.105155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:54.105155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:54.105155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:54.105956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:54.105956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:54.105956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:54.105956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:54.105956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:54.105956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:54.105956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:54.105956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:54.105956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:54.105956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:54.105956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 31 14:24:54.105956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:24:54.105956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 164/306] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:54.114295 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e7e8d-c97a-4fed-9297-72b6e79c0a44 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:54.137334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235820c-d77f-4711-97cd-73632ffbf5a6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:54.141499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:54.191848 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153167f-5fe8-49d7-846a-2af40b7e2cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:54.193470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153167f-5fe8-49d7-846a-2af40b7e2cdf None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:54.193944 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153167f-5fe8-49d7-846a-2af40b7e2cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:54.194693 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153167f-5fe8-49d7-846a-2af40b7e2cdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:54.195994 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153167f-5fe8-49d7-846a-2af40b7e2cdf None admin] RBAC: Adding query filter params (role.id=8655041bde1948a1be442c6239496c22, user.id=89cad0e7021f447dbb7268c90cfbcdb9, scope.system=all) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:54.196908 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153167f-5fe8-49d7-846a-2af40b7e2cdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:54.199128 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153167f-5fe8-49d7-846a-2af40b7e2cdf None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:54.214904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 143/307] 10.222.0.67 () {56 vars in 1308 bytes} [Tue May 31 14:24:53 2022] GET /identity/v3/role_assignments?user.id=89cad0e7021f447dbb7268c90cfbcdb9&scope.system=all&role.id=8655041bde1948a1be442c6239496c22 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:24:54.252785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e7e8d-c97a-4fed-9297-72b6e79c0a44 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:54.295196 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f396b1e9-2144-493e-8401-c615d55fd682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:54.296268 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396b1e9-2144-493e-8401-c615d55fd682 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:54.296779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396b1e9-2144-493e-8401-c615d55fd682 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:54.297260 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396b1e9-2144-493e-8401-c615d55fd682 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:54.298281 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f396b1e9-2144-493e-8401-c615d55fd682 None admin] RBAC: Adding query filter params (name=member) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:54.298850 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f396b1e9-2144-493e-8401-c615d55fd682 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:54.300499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f396b1e9-2144-493e-8401-c615d55fd682 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:54.310855 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 165/308] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:24:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:54.319822 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153167f-5fe8-49d7-846a-2af40b7e2cdf None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:54.432937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:54.437077 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153167f-5fe8-49d7-846a-2af40b7e2cdf None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:54.475016 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361704c-7774-448b-b680-eacc1cb500eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:54.475903 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361704c-7774-448b-b680-eacc1cb500eb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:54.476189 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361704c-7774-448b-b680-eacc1cb500eb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:54.476469 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361704c-7774-448b-b680-eacc1cb500eb None admin] PATH_INFO: `/v3/users/7411ede0d7d749b5aa52558188b3a3d9` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:54.477792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361704c-7774-448b-b680-eacc1cb500eb None admin] RBAC: Authorizing `identity:get_user(user_id=7411ede0d7d749b5aa52558188b3a3d9)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:54.479274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361704c-7774-448b-b680-eacc1cb500eb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:54.480923 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 144/309] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:54 2022] GET /identity/v3/users/7411ede0d7d749b5aa52558188b3a3d9 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:54.487701 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f396b1e9-2144-493e-8401-c615d55fd682 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:54.569889 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:54.595927 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444accd6-294a-4e1a-a233-66b45b2aa9ec None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:54.596251 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444accd6-294a-4e1a-a233-66b45b2aa9ec None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:54.596827 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444accd6-294a-4e1a-a233-66b45b2aa9ec None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:54.598492 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 145/310] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:54.599584 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f396b1e9-2144-493e-8401-c615d55fd682 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:54.603610 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d4b2a-baf5-4eae-96bc-1ac2e1f90cac None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:54.603900 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d4b2a-baf5-4eae-96bc-1ac2e1f90cac None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:54.604177 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d4b2a-baf5-4eae-96bc-1ac2e1f90cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:54.621326 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6b1d4b2a-baf5-4eae-96bc-1ac2e1f90cac None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:54.623212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6b1d4b2a-baf5-4eae-96bc-1ac2e1f90cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:54.641982 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019deda-d716-4b58-9cff-cf8e3616fad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:54.642990 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019deda-d716-4b58-9cff-cf8e3616fad3 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:54.643279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019deda-d716-4b58-9cff-cf8e3616fad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:54.643559 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019deda-d716-4b58-9cff-cf8e3616fad3 None admin] PATH_INFO: `/v3/system/users/7411ede0d7d749b5aa52558188b3a3d9/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:54.645059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019deda-d716-4b58-9cff-cf8e3616fad3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7411ede0d7d749b5aa52558188b3a3d9, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:54.646558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019deda-d716-4b58-9cff-cf8e3616fad3 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:54.657338 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 166/311] 10.222.0.67 () {58 vars in 1334 bytes} [Tue May 31 14:24:54 2022] PUT /identity/v3/system/users/7411ede0d7d749b5aa52558188b3a3d9/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:24:54.712578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 146/312] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 31 14:24:54.739454 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019deda-d716-4b58-9cff-cf8e3616fad3 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:54.853682 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019deda-d716-4b58-9cff-cf8e3616fad3 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:54.891786 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c9922-e4ee-4a34-a038-5c1d1d39edad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:54.892586 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c9922-e4ee-4a34-a038-5c1d1d39edad None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:54.892866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c9922-e4ee-4a34-a038-5c1d1d39edad None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:54.893142 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c9922-e4ee-4a34-a038-5c1d1d39edad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:54.904741 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c9922-e4ee-4a34-a038-5c1d1d39edad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:54.912455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-337c9922-e4ee-4a34-a038-5c1d1d39edad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 31 14:24:54.912455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:54.912455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:54.912455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:54.912455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:54.912455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:54.912455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:54.912455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:54.912455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:54.912455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:54.912455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:54.912455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:54.912455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:54.912455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:54.912455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:54.912455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:54.913315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:54.913315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:54.913315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:54.913315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:54.913315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:54.913315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:54.913315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:54.913315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:54.913315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:54.913315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:54.913315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:54.913315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:24:54.913315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:54.913315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:54.913315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:54.913315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:54.914277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:54.914277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:54.914277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:54.914277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:54.914277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:54.914277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:54.914277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:54.914277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:54.914277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:54.914277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:54.914277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 31 14:24:54.914277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:24:54.914277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 167/313] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:54.919972 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d4b2a-baf5-4eae-96bc-1ac2e1f90cac None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:55.021516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:55.027673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d4b2a-baf5-4eae-96bc-1ac2e1f90cac None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:55.063518 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0cfe95-362d-4ee8-9450-42463fe2d07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:55.064207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cfe95-362d-4ee8-9450-42463fe2d07e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:55.064495 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cfe95-362d-4ee8-9450-42463fe2d07e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:55.064776 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cfe95-362d-4ee8-9450-42463fe2d07e None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:55.065537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0cfe95-362d-4ee8-9450-42463fe2d07e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:55.065851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0cfe95-362d-4ee8-9450-42463fe2d07e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:55.067267 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0cfe95-362d-4ee8-9450-42463fe2d07e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:55.077208 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 147/314] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:24:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:55.082746 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c9922-e4ee-4a34-a038-5c1d1d39edad None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:55.086810 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:55.193204 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c9922-e4ee-4a34-a038-5c1d1d39edad None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:55.231121 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00812d5a-75f8-4db2-bca9-783f30aea01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:55.232035 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00812d5a-75f8-4db2-bca9-783f30aea01f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:55.232313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00812d5a-75f8-4db2-bca9-783f30aea01f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:55.232587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00812d5a-75f8-4db2-bca9-783f30aea01f None admin] PATH_INFO: `/v3/users/c41f416318cc4813877c7c3ea89bf8fd` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:55.233795 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00812d5a-75f8-4db2-bca9-783f30aea01f None admin] RBAC: Authorizing `identity:get_user(user_id=c41f416318cc4813877c7c3ea89bf8fd)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:55.235305 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00812d5a-75f8-4db2-bca9-783f30aea01f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:55.237002 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 168/315] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:55 2022] GET /identity/v3/users/c41f416318cc4813877c7c3ea89bf8fd => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:55.242471 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0cfe95-362d-4ee8-9450-42463fe2d07e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:55.344065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0cfe95-362d-4ee8-9450-42463fe2d07e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:55.378784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0be5c-095e-48c7-a9c1-849984c81814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:55.379445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0be5c-095e-48c7-a9c1-849984c81814 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:55.379741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0be5c-095e-48c7-a9c1-849984c81814 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:55.380022 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0be5c-095e-48c7-a9c1-849984c81814 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:55.421110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:55.453045 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0be5c-095e-48c7-a9c1-849984c81814 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:55.454516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0be5c-095e-48c7-a9c1-849984c81814 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:55.560567 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 148/316] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:24:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:24:55.565677 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00812d5a-75f8-4db2-bca9-783f30aea01f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:55.636617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:55.667604 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00812d5a-75f8-4db2-bca9-783f30aea01f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:55.702286 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d549990-53ea-4246-80ae-d212781d9ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:55.703112 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d549990-53ea-4246-80ae-d212781d9ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:55.703396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d549990-53ea-4246-80ae-d212781d9ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:55.703671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d549990-53ea-4246-80ae-d212781d9ae8 None admin] PATH_INFO: `/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:55.704847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d549990-53ea-4246-80ae-d212781d9ae8 None admin] RBAC: Authorizing `identity:get_project(project_id=bef8d3a846e34a8daf7eecb46a5fd2f8)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:55.706286 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d549990-53ea-4246-80ae-d212781d9ae8 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:55.708307 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 169/317] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:24:55 2022] GET /identity/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:55.713992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0be5c-095e-48c7-a9c1-849984c81814 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:55.772298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:55.814143 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0be5c-095e-48c7-a9c1-849984c81814 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:55.851634 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e7e2c-7ff7-4762-9154-667216b88616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:55.852297 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e7e2c-7ff7-4762-9154-667216b88616 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:55.852582 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e7e2c-7ff7-4762-9154-667216b88616 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:55.852858 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e7e2c-7ff7-4762-9154-667216b88616 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:55.853702 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e7e2c-7ff7-4762-9154-667216b88616 None admin] RBAC: Adding query filter params (role.id=8655041bde1948a1be442c6239496c22, scope.project.id=bef8d3a846e34a8daf7eecb46a5fd2f8, user.id=c41f416318cc4813877c7c3ea89bf8fd) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:55.854013 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e7e2c-7ff7-4762-9154-667216b88616 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:55.855474 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e7e2c-7ff7-4762-9154-667216b88616 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:55.862119 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 149/318] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:24:55 2022] GET /identity/v3/role_assignments?user.id=c41f416318cc4813877c7c3ea89bf8fd&scope.project.id=bef8d3a846e34a8daf7eecb46a5fd2f8&role.id=8655041bde1948a1be442c6239496c22 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:56.301075 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee477227-2d85-4994-a265-d1a2bceb482b None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:56.301594 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee477227-2d85-4994-a265-d1a2bceb482b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:56.301668 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee477227-2d85-4994-a265-d1a2bceb482b None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:56.303437 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 170/319] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:56.309231 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ceef15-54de-4936-bc7e-16c246223fcc None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:56.310308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ceef15-54de-4936-bc7e-16c246223fcc None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:56.310308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ceef15-54de-4936-bc7e-16c246223fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:56.328020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f8ceef15-54de-4936-bc7e-16c246223fcc None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:56.329201 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f8ceef15-54de-4936-bc7e-16c246223fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:56.395916 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 150/320] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:56.418270 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee477227-2d85-4994-a265-d1a2bceb482b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:56.521440 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee477227-2d85-4994-a265-d1a2bceb482b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:56.555391 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f66086-14c0-4d83-9575-4a5c661043e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:56.556196 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f66086-14c0-4d83-9575-4a5c661043e3 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:56.556486 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f66086-14c0-4d83-9575-4a5c661043e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:56.556761 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f66086-14c0-4d83-9575-4a5c661043e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:56.557925 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f66086-14c0-4d83-9575-4a5c661043e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:56.559442 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f66086-14c0-4d83-9575-4a5c661043e3 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:56.561222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 171/321] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:24:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:56.566864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ceef15-54de-4936-bc7e-16c246223fcc None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:56.621112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:56.624207 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20267d72-2800-4444-98cf-81a52ae8f520 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:56.624716 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20267d72-2800-4444-98cf-81a52ae8f520 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:56.625002 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20267d72-2800-4444-98cf-81a52ae8f520 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:56.626414 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 172/322] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:56.632585 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c85de6-5018-4181-9775-c27bbdaea535 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:56.632876 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c85de6-5018-4181-9775-c27bbdaea535 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:56.633334 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c85de6-5018-4181-9775-c27bbdaea535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:56.645739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:56.652405 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-08c85de6-5018-4181-9775-c27bbdaea535 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:56.654201 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-08c85de6-5018-4181-9775-c27bbdaea535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:56.676445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ceef15-54de-4936-bc7e-16c246223fcc None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:56.712888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579be10b-5dba-49c1-996d-04a3c9e2ef0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:56.713617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579be10b-5dba-49c1-996d-04a3c9e2ef0f None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:56.713923 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579be10b-5dba-49c1-996d-04a3c9e2ef0f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:56.714238 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579be10b-5dba-49c1-996d-04a3c9e2ef0f None admin] PATH_INFO: `/v3/groups` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:56.714979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579be10b-5dba-49c1-996d-04a3c9e2ef0f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:56.716975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579be10b-5dba-49c1-996d-04a3c9e2ef0f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:56.724232 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 173/323] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 31 14:24:56.741657 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 151/324] 10.222.0.67 () {60 vars in 1134 bytes} [Tue May 31 14:24:56 2022] POST /identity/v3/groups => generated 226 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:24:56.749602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c85de6-5018-4181-9775-c27bbdaea535 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:56.857193 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c85de6-5018-4181-9775-c27bbdaea535 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:56.895306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726eccc-b366-4427-ba37-6c529096ccc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:56.896175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726eccc-b366-4427-ba37-6c529096ccc4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:56.896465 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726eccc-b366-4427-ba37-6c529096ccc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:56.896744 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726eccc-b366-4427-ba37-6c529096ccc4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:56.899810 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:56.910582 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726eccc-b366-4427-ba37-6c529096ccc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:56.918470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-8726eccc-b366-4427-ba37-6c529096ccc4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 31 14:24:56.918470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:56.918470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:56.918470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:56.918470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:56.918470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:56.918470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:56.918470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:56.918470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:56.918470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:56.918470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:56.918470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:56.918470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:56.918470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:56.918470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:56.918470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:56.919430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:56.919430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:56.919430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:56.919430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:56.919430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:56.919430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:56.919430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:56.919430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:56.919430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:56.919430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:56.919430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:56.919430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:24:56.919430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:56.919430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:56.919430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:56.919430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:56.920292 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:56.920292 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:56.920292 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:56.920292 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:56.920292 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:56.920292 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:56.920292 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:56.920292 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:56.920292 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:56.920292 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:56.920292 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 31 14:24:56.920292 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:24:56.922980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 174/325] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:56.925700 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579be10b-5dba-49c1-996d-04a3c9e2ef0f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:56.976071 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e57ba9-6ae2-4fa5-9f68-5eb0a977f4f4 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:56.976369 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e57ba9-6ae2-4fa5-9f68-5eb0a977f4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:56.976649 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e57ba9-6ae2-4fa5-9f68-5eb0a977f4f4 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:56.978152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 175/326] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 31 14:24:56.983180 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd817f2a-324a-4fd4-8533-ebe52c4c8157 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:56.983470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd817f2a-324a-4fd4-8533-ebe52c4c8157 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:56.983747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd817f2a-324a-4fd4-8533-ebe52c4c8157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:57.003322 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-bd817f2a-324a-4fd4-8533-ebe52c4c8157 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:57.004547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-bd817f2a-324a-4fd4-8533-ebe52c4c8157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:57.048539 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579be10b-5dba-49c1-996d-04a3c9e2ef0f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:57.068831 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:57.093799 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 176/327] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 31 14:24:57.094407 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad772cb-ac73-4f50-b138-fabbcbcc1b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:57.095159 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad772cb-ac73-4f50-b138-fabbcbcc1b55 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:57.095441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad772cb-ac73-4f50-b138-fabbcbcc1b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:57.095713 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad772cb-ac73-4f50-b138-fabbcbcc1b55 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:57.096469 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad772cb-ac73-4f50-b138-fabbcbcc1b55 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:57.096775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad772cb-ac73-4f50-b138-fabbcbcc1b55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:57.098139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad772cb-ac73-4f50-b138-fabbcbcc1b55 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:57.107946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 152/328] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:24:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:24:57.114395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd817f2a-324a-4fd4-8533-ebe52c4c8157 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:57.119478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad772cb-ac73-4f50-b138-fabbcbcc1b55 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:57.244669 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd817f2a-324a-4fd4-8533-ebe52c4c8157 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:57.248230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad772cb-ac73-4f50-b138-fabbcbcc1b55 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:57.279996 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:57.289479 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba05374-7906-43d5-b22f-7528bbb495a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:57.290186 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1c86f-7c71-4d84-8135-c58a3c1ce66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:57.290359 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba05374-7906-43d5-b22f-7528bbb495a4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:57.290620 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba05374-7906-43d5-b22f-7528bbb495a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:57.290920 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba05374-7906-43d5-b22f-7528bbb495a4 None admin] PATH_INFO: `/v3/users/89cad0e7021f447dbb7268c90cfbcdb9` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:57.291751 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1c86f-7c71-4d84-8135-c58a3c1ce66d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:57.292098 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba05374-7906-43d5-b22f-7528bbb495a4 None admin] RBAC: Authorizing `identity:get_user(user_id=89cad0e7021f447dbb7268c90cfbcdb9)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:57.292411 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1c86f-7c71-4d84-8135-c58a3c1ce66d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:57.292860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1c86f-7c71-4d84-8135-c58a3c1ce66d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:57.293571 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba05374-7906-43d5-b22f-7528bbb495a4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:57.295274 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 177/329] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:57 2022] GET /identity/v3/users/89cad0e7021f447dbb7268c90cfbcdb9 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:57.300778 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba05374-7906-43d5-b22f-7528bbb495a4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:57.304926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb1c86f-7c71-4d84-8135-c58a3c1ce66d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:24:57.312853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-0bb1c86f-7c71-4d84-8135-c58a3c1ce66d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 31 14:24:57.312853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:24:57.312853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:24:57.312853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:24:57.312853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:24:57.312853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:24:57.312853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:24:57.312853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:24:57.312853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:24:57.312853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:24:57.312853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:24:57.312853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:24:57.312853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:24:57.312853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:24:57.312853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:24:57.312853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:24:57.314158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:24:57.314158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:24:57.314158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:24:57.314158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:24:57.314158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:24:57.314158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:24:57.314158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:24:57.314158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:24:57.314158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:24:57.314158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:24:57.314158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:24:57.314158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:24:57.314158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:24:57.314158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:24:57.314158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:24:57.314158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:24:57.314970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:24:57.314970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:24:57.314970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:24:57.314970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:24:57.314970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:24:57.314970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:24:57.314970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:24:57.314970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:24:57.314970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:24:57.314970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:24:57.314970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 31 14:24:57.314970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:24:57.314970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 153/330] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:24:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:24:57.320009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1c86f-7c71-4d84-8135-c58a3c1ce66d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:57.357759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:57.428669 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba05374-7906-43d5-b22f-7528bbb495a4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:57.441465 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1c86f-7c71-4d84-8135-c58a3c1ce66d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:57.473818 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df746d6c-fd68-466a-830d-52a631a038c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:57.476415 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df746d6c-fd68-466a-830d-52a631a038c9 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:57.476907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df746d6c-fd68-466a-830d-52a631a038c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:57.477293 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df746d6c-fd68-466a-830d-52a631a038c9 None admin] PATH_INFO: `/v3/system/users/89cad0e7021f447dbb7268c90cfbcdb9/roles/8655041bde1948a1be442c6239496c22` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:57.479088 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df746d6c-fd68-466a-830d-52a631a038c9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=89cad0e7021f447dbb7268c90cfbcdb9, role_id=8655041bde1948a1be442c6239496c22)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:57.479962 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025cf85-f180-42b8-8299-79b5d0177760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:57.480784 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df746d6c-fd68-466a-830d-52a631a038c9 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:57.480916 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025cf85-f180-42b8-8299-79b5d0177760 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:57.481396 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025cf85-f180-42b8-8299-79b5d0177760 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:57.481810 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025cf85-f180-42b8-8299-79b5d0177760 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:57.482903 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025cf85-f180-42b8-8299-79b5d0177760 None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:57.483350 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025cf85-f180-42b8-8299-79b5d0177760 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:57.484972 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025cf85-f180-42b8-8299-79b5d0177760 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:57.494572 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 154/331] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:24:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:24:57.494970 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 178/332] 10.222.0.67 () {58 vars in 1334 bytes} [Tue May 31 14:24:57 2022] PUT /identity/v3/system/users/89cad0e7021f447dbb7268c90cfbcdb9/roles/8655041bde1948a1be442c6239496c22 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:24:57.500709 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df746d6c-fd68-466a-830d-52a631a038c9 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:57.570634 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:57.618124 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df746d6c-fd68-466a-830d-52a631a038c9 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:57.657106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf18a5-12cf-4f71-bb30-98d7712f72fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:57.658004 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf18a5-12cf-4f71-bb30-98d7712f72fe None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:57.658284 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf18a5-12cf-4f71-bb30-98d7712f72fe None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:57.658555 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf18a5-12cf-4f71-bb30-98d7712f72fe None admin] PATH_INFO: `/v3/users/7411ede0d7d749b5aa52558188b3a3d9` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:57.659832 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf18a5-12cf-4f71-bb30-98d7712f72fe None admin] RBAC: Authorizing `identity:get_user(user_id=7411ede0d7d749b5aa52558188b3a3d9)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:57.661305 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf18a5-12cf-4f71-bb30-98d7712f72fe None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:57.663352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 179/333] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:24:57 2022] GET /identity/v3/users/7411ede0d7d749b5aa52558188b3a3d9 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:24:57.669318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025cf85-f180-42b8-8299-79b5d0177760 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:24:57.777745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:24:57.784812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025cf85-f180-42b8-8299-79b5d0177760 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:24:57.819862 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31a628-60e7-42a8-bb82-e7726b296025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:24:57.820524 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31a628-60e7-42a8-bb82-e7726b296025 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:57.820809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31a628-60e7-42a8-bb82-e7726b296025 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:57.821077 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31a628-60e7-42a8-bb82-e7726b296025 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:57.821925 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31a628-60e7-42a8-bb82-e7726b296025 None admin] RBAC: Adding query filter params (role.id=9ce8fdecf52d482d894fdeec4f530624, user.id=7411ede0d7d749b5aa52558188b3a3d9, scope.system=all) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:24:57.822236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31a628-60e7-42a8-bb82-e7726b296025 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:24:57.823699 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31a628-60e7-42a8-bb82-e7726b296025 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:24:57.837223 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 155/334] 10.222.0.67 () {56 vars in 1308 bytes} [Tue May 31 14:24:57 2022] GET /identity/v3/role_assignments?user.id=7411ede0d7d749b5aa52558188b3a3d9&scope.system=all&role.id=9ce8fdecf52d482d894fdeec4f530624 => generated 510 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:24:59.717688 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e5b27-a88c-4e35-87a5-e036b3d47eb8 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:59.718779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e5b27-a88c-4e35-87a5-e036b3d47eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:59.719270 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e5b27-a88c-4e35-87a5-e036b3d47eb8 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:59.721207 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 180/335] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:24:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:24:59.728703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925aee57-aef6-44c9-a208-312021c52fb4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:24:59.729209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925aee57-aef6-44c9-a208-312021c52fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:24:59.729536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925aee57-aef6-44c9-a208-312021c52fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:24:59.751644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-925aee57-aef6-44c9-a208-312021c52fb4 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:24:59.752951 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-925aee57-aef6-44c9-a208-312021c52fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:24:59.837275 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 156/336] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:24:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:24:59.875756 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e5b27-a88c-4e35-87a5-e036b3d47eb8 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:00.016421 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e5b27-a88c-4e35-87a5-e036b3d47eb8 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:00.048011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:00.061982 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a3d3f-5067-41a7-a48a-f8f85a2f9304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:00.062837 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a3d3f-5067-41a7-a48a-f8f85a2f9304 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:00.063128 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a3d3f-5067-41a7-a48a-f8f85a2f9304 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:00.063406 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a3d3f-5067-41a7-a48a-f8f85a2f9304 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:00.077191 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5a3d3f-5067-41a7-a48a-f8f85a2f9304 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:00.081263 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e489ab0-061c-4f53-a688-e21c64fe73fa None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:00.081816 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e489ab0-061c-4f53-a688-e21c64fe73fa None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:00.082300 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e489ab0-061c-4f53-a688-e21c64fe73fa None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:00.084128 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 157/337] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:00.086717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-4f5a3d3f-5067-41a7-a48a-f8f85a2f9304 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:25:00.086717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:00.086717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:00.086717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:00.086717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:00.086717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:00.086717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:00.086717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:00.086717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:00.086717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:00.086717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:00.086717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:00.086717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:00.086717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:00.086717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:00.086717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:00.087661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:00.087661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:00.087661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:00.087661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:00.087661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:00.087661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:00.087661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:00.087661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:00.087661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:00.087661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:00.087661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:00.087661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:00.087661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:00.087661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:00.087661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:00.087661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:00.088495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:00.088495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:00.088495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:00.088495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:00.088495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:00.088495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:00.088495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:00.088495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:00.088495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:00.088495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:00.088495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:25:00.088495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:00.088495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 181/338] 10.222.0.67 () {56 vars in 1114 bytes} [Tue May 31 14:24:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:00.093578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a3d3f-5067-41a7-a48a-f8f85a2f9304 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:00.094109 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03f436-75a3-4f6e-a04b-4b2a3071d125 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:00.094612 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03f436-75a3-4f6e-a04b-4b2a3071d125 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:00.095190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03f436-75a3-4f6e-a04b-4b2a3071d125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:00.127687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-bb03f436-75a3-4f6e-a04b-4b2a3071d125 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:00.129625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bb03f436-75a3-4f6e-a04b-4b2a3071d125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:00.228208 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a3d3f-5067-41a7-a48a-f8f85a2f9304 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:00.253745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 158/339] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 31 14:25:00.267014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5ff4f-fb0f-4d34-a408-4be5371da6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:00.267627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5ff4f-fb0f-4d34-a408-4be5371da6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:00.267915 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5ff4f-fb0f-4d34-a408-4be5371da6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:00.268196 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5ff4f-fb0f-4d34-a408-4be5371da6e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:00.268956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5ff4f-fb0f-4d34-a408-4be5371da6e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:00.269269 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5ff4f-fb0f-4d34-a408-4be5371da6e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:00.270670 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5ff4f-fb0f-4d34-a408-4be5371da6e9 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:00.280146 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 182/340] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:25:00.280338 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d03d1f-7b6b-484b-8066-1e23cfd6efd1 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:00.284726 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d03d1f-7b6b-484b-8066-1e23cfd6efd1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:00.284726 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d03d1f-7b6b-484b-8066-1e23cfd6efd1 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:00.284726 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 159/341] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 31 14:25:00.286233 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5ff4f-fb0f-4d34-a408-4be5371da6e9 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:00.296669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33724b7-1f55-4a01-9bc1-ef40a37763fe None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:00.296963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33724b7-1f55-4a01-9bc1-ef40a37763fe None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:00.297248 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33724b7-1f55-4a01-9bc1-ef40a37763fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:00.323334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f33724b7-1f55-4a01-9bc1-ef40a37763fe None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:00.324533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f33724b7-1f55-4a01-9bc1-ef40a37763fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:00.348761 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:00.363098 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:00.400298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5ff4f-fb0f-4d34-a408-4be5371da6e9 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:00.437731 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bdb7e9-ae64-4d2f-9e6d-e35b0cdddba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:00.438766 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdb7e9-ae64-4d2f-9e6d-e35b0cdddba1 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:00.439252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdb7e9-ae64-4d2f-9e6d-e35b0cdddba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:00.439849 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdb7e9-ae64-4d2f-9e6d-e35b0cdddba1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:00.517962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bdb7e9-ae64-4d2f-9e6d-e35b0cdddba1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:00.519851 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bdb7e9-ae64-4d2f-9e6d-e35b0cdddba1 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:00.523297 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 160/342] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:00.524616 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33724b7-1f55-4a01-9bc1-ef40a37763fe None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:00.603051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:00.629602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 183/343] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 31 14:25:00.630620 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bdb7e9-ae64-4d2f-9e6d-e35b0cdddba1 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:00.746272 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bdb7e9-ae64-4d2f-9e6d-e35b0cdddba1 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:00.785703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33724b7-1f55-4a01-9bc1-ef40a37763fe None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:00.794337 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9e1a1-41c7-4654-b8a8-f0a06d388301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:00.795391 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9e1a1-41c7-4654-b8a8-f0a06d388301 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:00.795833 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9e1a1-41c7-4654-b8a8-f0a06d388301 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:00.796274 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9e1a1-41c7-4654-b8a8-f0a06d388301 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:00.822259 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a9e1a1-41c7-4654-b8a8-f0a06d388301 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:00.831912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-05a9e1a1-41c7-4654-b8a8-f0a06d388301 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 31 14:25:00.831912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:00.831912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:00.831912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:00.831912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:00.831912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:00.831912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:00.831912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:00.831912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:00.831912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:00.831912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:00.831912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:00.831912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:00.831912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:00.831912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:00.831912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:00.832828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:00.832828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:00.832828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:00.832828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:00.832828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:00.832828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:00.832828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:00.832828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:00.832828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:00.832828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:00.832828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:00.832828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:00.832828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:00.832828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:00.832828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:00.832828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:00.833612 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:00.833612 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:00.833612 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:00.833612 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:00.833612 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:00.833612 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:00.833612 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:00.833612 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:00.833612 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:00.833612 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:00.833612 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 31 14:25:00.833612 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:00.834220 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d14bc-6612-457c-8f9c-55dcd2b4f69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:00.834220 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d14bc-6612-457c-8f9c-55dcd2b4f69e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:00.834220 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d14bc-6612-457c-8f9c-55dcd2b4f69e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:00.834220 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d14bc-6612-457c-8f9c-55dcd2b4f69e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:00.840724 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 184/344] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:25:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:00.841791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9e1a1-41c7-4654-b8a8-f0a06d388301 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:00.847831 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d14bc-6612-457c-8f9c-55dcd2b4f69e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:00.858952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-4e9d14bc-6612-457c-8f9c-55dcd2b4f69e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 31 14:25:00.858952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:00.858952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:00.858952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:00.858952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:00.858952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:00.858952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:00.858952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:00.858952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:00.858952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:00.858952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:00.858952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:00.858952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:00.858952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:00.858952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:00.858952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:00.859958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:00.859958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:00.859958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:00.859958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:00.859958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:00.859958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:00.859958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:00.859958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:00.859958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:00.859958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:00.859958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:00.859958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:00.859958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:00.859958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:00.859958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:00.859958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:00.860809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:00.860809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:00.860809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:00.860809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:00.860809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:00.860809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:00.860809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:00.860809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:00.860809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:00.860809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:00.860809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 31 14:25:00.860809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:00.860809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 161/345] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:25:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:00.861551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d14bc-6612-457c-8f9c-55dcd2b4f69e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:00.867473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:00.873823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:00.997987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9e1a1-41c7-4654-b8a8-f0a06d388301 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:01.002634 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d14bc-6612-457c-8f9c-55dcd2b4f69e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:01.046406 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd3c5b-1b71-4abd-8c5f-6e1510d4a4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:01.047666 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd3c5b-1b71-4abd-8c5f-6e1510d4a4bf None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:01.048150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd3c5b-1b71-4abd-8c5f-6e1510d4a4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:01.048623 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd3c5b-1b71-4abd-8c5f-6e1510d4a4bf None admin] PATH_INFO: `/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:01.050021 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd3c5b-1b71-4abd-8c5f-6e1510d4a4bf None admin] RBAC: Authorizing `identity:get_project(project_id=e3f26f512b354f6ebdbf6c8d3cc3e472)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:01.050149 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df086def-298c-4640-a405-eb89914c42bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:01.050792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df086def-298c-4640-a405-eb89914c42bf None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:01.051082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df086def-298c-4640-a405-eb89914c42bf None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:01.051363 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df086def-298c-4640-a405-eb89914c42bf None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:01.052196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df086def-298c-4640-a405-eb89914c42bf None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:01.052508 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df086def-298c-4640-a405-eb89914c42bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:01.052968 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd3c5b-1b71-4abd-8c5f-6e1510d4a4bf None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:01.053876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df086def-298c-4640-a405-eb89914c42bf None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:01.055718 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 185/346] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:25:00 2022] GET /identity/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:01.056818 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd3c5b-1b71-4abd-8c5f-6e1510d4a4bf None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:01.065196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 162/347] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:25:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:01.067100 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5216221-d838-45ee-a850-45d4041249c7 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:01.067100 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5216221-d838-45ee-a850-45d4041249c7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:01.067100 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5216221-d838-45ee-a850-45d4041249c7 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:01.068414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 163/348] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:01.069122 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5216221-d838-45ee-a850-45d4041249c7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:01.160402 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:01.168203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:01.196308 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd3c5b-1b71-4abd-8c5f-6e1510d4a4bf None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:01.215719 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5216221-d838-45ee-a850-45d4041249c7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:01.243231 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953de72d-356c-4ff1-a976-0b510e2230a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:01.244074 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953de72d-356c-4ff1-a976-0b510e2230a7 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:01.244734 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953de72d-356c-4ff1-a976-0b510e2230a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:01.245116 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953de72d-356c-4ff1-a976-0b510e2230a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:01.245980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953de72d-356c-4ff1-a976-0b510e2230a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:01.246499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953de72d-356c-4ff1-a976-0b510e2230a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:01.248466 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953de72d-356c-4ff1-a976-0b510e2230a7 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:01.258436 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 186/349] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:25:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:01.262068 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953de72d-356c-4ff1-a976-0b510e2230a7 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:01.263176 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f256c-f647-48f4-b9fe-0af327d37c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:01.265130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f256c-f647-48f4-b9fe-0af327d37c75 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:01.265559 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f256c-f647-48f4-b9fe-0af327d37c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:01.265977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f256c-f647-48f4-b9fe-0af327d37c75 None admin] PATH_INFO: `/v3/groups/93ca2d3407df4b40a423023aec0efbbe` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:01.274744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1f256c-f647-48f4-b9fe-0af327d37c75 None admin] RBAC: Authorizing `identity:get_group(group_id=93ca2d3407df4b40a423023aec0efbbe)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:01.276941 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1f256c-f647-48f4-b9fe-0af327d37c75 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:01.279183 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 164/350] 10.222.0.67 () {56 vars in 1180 bytes} [Tue May 31 14:25:01 2022] GET /identity/v3/groups/93ca2d3407df4b40a423023aec0efbbe => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:01.280650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca206c7-31b7-4078-9791-1d15a2220cfe None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:01.281068 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca206c7-31b7-4078-9791-1d15a2220cfe None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:01.281498 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca206c7-31b7-4078-9791-1d15a2220cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:01.307502 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3ca206c7-31b7-4078-9791-1d15a2220cfe None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:01.308935 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3ca206c7-31b7-4078-9791-1d15a2220cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:01.388146 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953de72d-356c-4ff1-a976-0b510e2230a7 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:01.393856 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 165/351] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:01.399476 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca206c7-31b7-4078-9791-1d15a2220cfe None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:01.449227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36b0d3-03bc-458a-87b7-24addb1a035d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:01.450088 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36b0d3-03bc-458a-87b7-24addb1a035d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:01.450475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36b0d3-03bc-458a-87b7-24addb1a035d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:01.450965 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36b0d3-03bc-458a-87b7-24addb1a035d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:01.463454 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:01.475022 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:01.515834 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca206c7-31b7-4078-9791-1d15a2220cfe None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:01.545367 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36b0d3-03bc-458a-87b7-24addb1a035d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:01.546979 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36b0d3-03bc-458a-87b7-24addb1a035d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:01.555633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc84533-a7dd-470f-a853-8696a137a42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:01.556383 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc84533-a7dd-470f-a853-8696a137a42b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:01.556735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc84533-a7dd-470f-a853-8696a137a42b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:01.557085 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc84533-a7dd-470f-a853-8696a137a42b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:01.646415 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc84533-a7dd-470f-a853-8696a137a42b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:01.648362 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc84533-a7dd-470f-a853-8696a137a42b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:01.680724 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 187/352] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:25:01.682096 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36b0d3-03bc-458a-87b7-24addb1a035d None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:01.721919 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:01.736487 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:01.770688 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 166/353] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:25:01.771887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc84533-a7dd-470f-a853-8696a137a42b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:01.823934 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36b0d3-03bc-458a-87b7-24addb1a035d None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:01.878748 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1117d-b564-49c8-9735-cc7d21497881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:01.879682 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1117d-b564-49c8-9735-cc7d21497881 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:01.880109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1117d-b564-49c8-9735-cc7d21497881 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:01.880531 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1117d-b564-49c8-9735-cc7d21497881 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:01.881381 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1117d-b564-49c8-9735-cc7d21497881 None admin] RBAC: Adding query filter params (group.id=93ca2d3407df4b40a423023aec0efbbe, role.id=1b9c18d4b8b540bf9825080748cec3bc, scope.project.id=e3f26f512b354f6ebdbf6c8d3cc3e472) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:01.881699 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1117d-b564-49c8-9735-cc7d21497881 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:01.883197 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1117d-b564-49c8-9735-cc7d21497881 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:01.890101 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 188/354] 10.222.0.67 () {56 vars in 1376 bytes} [Tue May 31 14:25:01 2022] GET /identity/v3/role_assignments?group.id=93ca2d3407df4b40a423023aec0efbbe&scope.project.id=e3f26f512b354f6ebdbf6c8d3cc3e472&role.id=1b9c18d4b8b540bf9825080748cec3bc => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:01.891082 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8a363-c6ff-47d4-860c-bb9c9f9b0c06 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:01.891375 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8a363-c6ff-47d4-860c-bb9c9f9b0c06 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:01.891712 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8a363-c6ff-47d4-860c-bb9c9f9b0c06 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:01.893109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 189/355] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:01.897420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8a363-c6ff-47d4-860c-bb9c9f9b0c06 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:01.907097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc84533-a7dd-470f-a853-8696a137a42b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:01.952136 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4354158-bc55-468f-9f1b-1cf358861926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:01.953361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4354158-bc55-468f-9f1b-1cf358861926 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:01.953831 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4354158-bc55-468f-9f1b-1cf358861926 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:01.954286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4354158-bc55-468f-9f1b-1cf358861926 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:01.972000 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4354158-bc55-468f-9f1b-1cf358861926 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:01.980879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-d4354158-bc55-468f-9f1b-1cf358861926 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 31 14:25:01.980879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:01.980879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:01.980879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:01.980879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:01.980879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:01.980879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:01.980879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:01.980879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:01.980879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:01.980879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:01.980879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:01.980879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:01.980879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:01.980879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:01.980879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:01.981913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:01.981913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:01.981913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:01.981913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:01.981913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:01.981913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:01.981913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:01.981913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:01.981913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:01.981913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:01.981913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:01.981913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:01.981913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:01.981913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:01.981913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:01.981913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:01.982740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:01.982740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:01.982740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:01.982740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:01.982740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:01.982740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:01.982740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:01.982740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:01.982740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:01.982740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:01.982740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 31 14:25:01.982740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:01.985257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 167/356] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:25:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:01.986330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4354158-bc55-468f-9f1b-1cf358861926 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:02.015368 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:02.019558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8a363-c6ff-47d4-860c-bb9c9f9b0c06 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:02.021719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:02.067332 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ff12e-174d-4d5d-b827-34f630439452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:02.068884 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ff12e-174d-4d5d-b827-34f630439452 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:02.069172 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ff12e-174d-4d5d-b827-34f630439452 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:02.069454 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ff12e-174d-4d5d-b827-34f630439452 None admin] PATH_INFO: `/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:02.070866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ff12e-174d-4d5d-b827-34f630439452 None admin] RBAC: Authorizing `identity:get_project(project_id=e3f26f512b354f6ebdbf6c8d3cc3e472)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:02.072210 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ff12e-174d-4d5d-b827-34f630439452 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:02.073956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 190/357] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:25:01 2022] GET /identity/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:02.075726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f11c89-ed90-4e25-bcd6-18daea5b7057 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:02.075978 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f11c89-ed90-4e25-bcd6-18daea5b7057 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:02.076262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f11c89-ed90-4e25-bcd6-18daea5b7057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:02.096573 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-f5f11c89-ed90-4e25-bcd6-18daea5b7057 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:02.099224 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f5f11c89-ed90-4e25-bcd6-18daea5b7057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:02.127463 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4354158-bc55-468f-9f1b-1cf358861926 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:02.167872 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99495094-41f6-4fcb-835d-c951780e0297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:02.168707 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99495094-41f6-4fcb-835d-c951780e0297 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:02.168990 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99495094-41f6-4fcb-835d-c951780e0297 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:02.169262 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99495094-41f6-4fcb-835d-c951780e0297 None admin] PATH_INFO: `/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:02.170469 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99495094-41f6-4fcb-835d-c951780e0297 None admin] RBAC: Authorizing `identity:get_project(project_id=bef8d3a846e34a8daf7eecb46a5fd2f8)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:02.171946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99495094-41f6-4fcb-835d-c951780e0297 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:02.173857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 168/358] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:25:01 2022] GET /identity/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:02.174495 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99495094-41f6-4fcb-835d-c951780e0297 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:02.192710 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 191/359] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:02.194493 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15400c5a-ebfe-447a-ae0b-5834547cbd93 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:02.195028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15400c5a-ebfe-447a-ae0b-5834547cbd93 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:02.195507 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15400c5a-ebfe-447a-ae0b-5834547cbd93 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:02.197005 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 192/360] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:02.197902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15400c5a-ebfe-447a-ae0b-5834547cbd93 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:02.297587 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:02.316702 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99495094-41f6-4fcb-835d-c951780e0297 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:02.319878 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:02.325912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15400c5a-ebfe-447a-ae0b-5834547cbd93 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:02.365488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91be289a-d2c6-4ed6-a0b4-aece553cdef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:02.366463 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be289a-d2c6-4ed6-a0b4-aece553cdef3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:02.367044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be289a-d2c6-4ed6-a0b4-aece553cdef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:02.367469 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be289a-d2c6-4ed6-a0b4-aece553cdef3 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:02.368373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91be289a-d2c6-4ed6-a0b4-aece553cdef3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:02.369036 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91be289a-d2c6-4ed6-a0b4-aece553cdef3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:02.370073 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d69c0-5986-4fa5-b2b3-6f043c8370bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:02.370783 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91be289a-d2c6-4ed6-a0b4-aece553cdef3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:02.370963 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d69c0-5986-4fa5-b2b3-6f043c8370bc None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:02.371251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d69c0-5986-4fa5-b2b3-6f043c8370bc None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:02.371532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d69c0-5986-4fa5-b2b3-6f043c8370bc None admin] PATH_INFO: `/v3/groups/93ca2d3407df4b40a423023aec0efbbe` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:02.373441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d69c0-5986-4fa5-b2b3-6f043c8370bc None admin] RBAC: Authorizing `identity:get_group(group_id=93ca2d3407df4b40a423023aec0efbbe)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:02.375311 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d69c0-5986-4fa5-b2b3-6f043c8370bc None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:02.376957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 193/361] 10.222.0.67 () {56 vars in 1180 bytes} [Tue May 31 14:25:02 2022] GET /identity/v3/groups/93ca2d3407df4b40a423023aec0efbbe => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:02.378669 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d69c0-5986-4fa5-b2b3-6f043c8370bc None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:02.382072 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 169/362] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:25:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:02.383550 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acd8ac-1625-4a14-8944-9767b569c678 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:02.383993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acd8ac-1625-4a14-8944-9767b569c678 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:02.384435 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acd8ac-1625-4a14-8944-9767b569c678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:02.407491 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-26acd8ac-1625-4a14-8944-9767b569c678 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:02.409016 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-26acd8ac-1625-4a14-8944-9767b569c678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:02.507617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 170/363] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:02.508983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d69c0-5986-4fa5-b2b3-6f043c8370bc None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:02.511468 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26acd8ac-1625-4a14-8944-9767b569c678 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:02.555222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85544caa-e4f9-4a69-bec9-0761bd53f49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:02.556139 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85544caa-e4f9-4a69-bec9-0761bd53f49e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:02.556428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85544caa-e4f9-4a69-bec9-0761bd53f49e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:02.556706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85544caa-e4f9-4a69-bec9-0761bd53f49e None admin] PATH_INFO: `/v3/groups/93ca2d3407df4b40a423023aec0efbbe` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:02.557915 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85544caa-e4f9-4a69-bec9-0761bd53f49e None admin] RBAC: Authorizing `identity:get_group(group_id=93ca2d3407df4b40a423023aec0efbbe)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:02.560479 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85544caa-e4f9-4a69-bec9-0761bd53f49e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:02.566668 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 194/364] 10.222.0.67 () {56 vars in 1180 bytes} [Tue May 31 14:25:02 2022] GET /identity/v3/groups/93ca2d3407df4b40a423023aec0efbbe => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:02.566668 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85544caa-e4f9-4a69-bec9-0761bd53f49e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:02.608053 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:02.614978 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:02.647488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26acd8ac-1625-4a14-8944-9767b569c678 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:02.691013 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c5712-044e-4132-861f-671348c9925f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:02.692232 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c5712-044e-4132-861f-671348c9925f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:02.692831 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c5712-044e-4132-861f-671348c9925f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:02.693444 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c5712-044e-4132-861f-671348c9925f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:02.696322 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85544caa-e4f9-4a69-bec9-0761bd53f49e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:02.705590 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c5712-044e-4132-861f-671348c9925f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:02.714584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-b05c5712-044e-4132-861f-671348c9925f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:25:02.714584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:02.714584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:02.714584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:02.714584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:02.714584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:02.714584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:02.714584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:02.714584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:02.714584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:02.714584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:02.714584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:02.714584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:02.714584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:02.714584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:02.714584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:02.715627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:02.715627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:02.715627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:02.715627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:02.715627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:02.715627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:02.715627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:02.715627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:02.715627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:02.715627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:02.715627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:02.715627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:02.715627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:02.715627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:02.715627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:02.715627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:02.716433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:02.716433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:02.716433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:02.716433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:02.716433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:02.716433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:02.716433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:02.716433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:02.716433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:02.716433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:02.716433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:25:02.716433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:02.719205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 171/365] 10.222.0.67 () {56 vars in 1114 bytes} [Tue May 31 14:25:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:02.720233 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c5712-044e-4132-861f-671348c9925f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:02.740013 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d404a-7e38-4cc4-b91f-d5799d53a3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:02.741532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d404a-7e38-4cc4-b91f-d5799d53a3fc None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:02.741532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d404a-7e38-4cc4-b91f-d5799d53a3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:02.741532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d404a-7e38-4cc4-b91f-d5799d53a3fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:02.742067 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d404a-7e38-4cc4-b91f-d5799d53a3fc None admin] RBAC: Adding query filter params (group.id=93ca2d3407df4b40a423023aec0efbbe, role.id=9ce8fdecf52d482d894fdeec4f530624, scope.project.id=e3f26f512b354f6ebdbf6c8d3cc3e472) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:02.742382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d404a-7e38-4cc4-b91f-d5799d53a3fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:02.743952 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d404a-7e38-4cc4-b91f-d5799d53a3fc None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:02.750602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 195/366] 10.222.0.67 () {56 vars in 1376 bytes} [Tue May 31 14:25:02 2022] GET /identity/v3/role_assignments?group.id=93ca2d3407df4b40a423023aec0efbbe&scope.project.id=e3f26f512b354f6ebdbf6c8d3cc3e472&role.id=9ce8fdecf52d482d894fdeec4f530624 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:02.751534 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d404a-7e38-4cc4-b91f-d5799d53a3fc None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:02.873533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c5712-044e-4132-861f-671348c9925f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:02.879029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:02.884003 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d404a-7e38-4cc4-b91f-d5799d53a3fc None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:02.912312 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de8ad1-ab24-47cb-bac1-8165d1ebe41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:02.913205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de8ad1-ab24-47cb-bac1-8165d1ebe41d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:02.913580 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de8ad1-ab24-47cb-bac1-8165d1ebe41d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:02.913762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de8ad1-ab24-47cb-bac1-8165d1ebe41d None admin] PATH_INFO: `/v3/users/89cad0e7021f447dbb7268c90cfbcdb9` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:02.915169 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de8ad1-ab24-47cb-bac1-8165d1ebe41d None admin] RBAC: Authorizing `identity:get_user(user_id=89cad0e7021f447dbb7268c90cfbcdb9)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:02.916879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de8ad1-ab24-47cb-bac1-8165d1ebe41d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:02.919141 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 172/367] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:25:02 2022] GET /identity/v3/users/89cad0e7021f447dbb7268c90cfbcdb9 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:02.920021 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de8ad1-ab24-47cb-bac1-8165d1ebe41d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:02.929846 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9eb677-8da0-484b-a358-d6c0ecfbb080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:02.931234 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9eb677-8da0-484b-a358-d6c0ecfbb080 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:02.931745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9eb677-8da0-484b-a358-d6c0ecfbb080 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:02.932224 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9eb677-8da0-484b-a358-d6c0ecfbb080 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:02.933371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:02.946885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9eb677-8da0-484b-a358-d6c0ecfbb080 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:02.962257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-7a9eb677-8da0-484b-a358-d6c0ecfbb080 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:25:02.962257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:02.962257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:02.962257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:02.962257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:02.962257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:02.962257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:02.962257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:02.962257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:02.962257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:02.962257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:02.962257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:02.962257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:02.962257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:02.962257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:02.962257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:02.963279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:02.963279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:02.963279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:02.963279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:02.963279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:02.963279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:02.963279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:02.963279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:02.963279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:02.963279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:02.963279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:02.963279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:02.963279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:02.963279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:02.963279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:02.963279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:02.964092 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:02.964092 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:02.964092 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:02.964092 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:02.964092 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:02.964092 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:02.964092 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:02.964092 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:02.964092 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:02.964092 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:02.964092 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:25:02.964092 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:02.967643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 196/368] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:25:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:02.968838 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9eb677-8da0-484b-a358-d6c0ecfbb080 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:03.052735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de8ad1-ab24-47cb-bac1-8165d1ebe41d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:03.097320 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9eb677-8da0-484b-a358-d6c0ecfbb080 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:03.099740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0dd82-7232-431e-ab9c-214073591a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:03.100835 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0dd82-7232-431e-ab9c-214073591a30 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:03.100835 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0dd82-7232-431e-ab9c-214073591a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:03.101053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0dd82-7232-431e-ab9c-214073591a30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:03.102051 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0dd82-7232-431e-ab9c-214073591a30 None admin] RBAC: Adding query filter params (group.id=93ca2d3407df4b40a423023aec0efbbe, role.id=9ce8fdecf52d482d894fdeec4f530624, scope.project.id=bef8d3a846e34a8daf7eecb46a5fd2f8) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:03.102761 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0dd82-7232-431e-ab9c-214073591a30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:03.104540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0dd82-7232-431e-ab9c-214073591a30 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:03.134118 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 173/369] 10.222.0.67 () {56 vars in 1376 bytes} [Tue May 31 14:25:02 2022] GET /identity/v3/role_assignments?group.id=93ca2d3407df4b40a423023aec0efbbe&scope.project.id=bef8d3a846e34a8daf7eecb46a5fd2f8&role.id=9ce8fdecf52d482d894fdeec4f530624 => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:03.137104 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0dd82-7232-431e-ab9c-214073591a30 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:03.208254 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576d1f0-a952-4f58-adb2-8a86d074325a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:03.213358 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576d1f0-a952-4f58-adb2-8a86d074325a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:03.213358 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576d1f0-a952-4f58-adb2-8a86d074325a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:03.213358 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576d1f0-a952-4f58-adb2-8a86d074325a None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:03.213358 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576d1f0-a952-4f58-adb2-8a86d074325a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:03.213358 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576d1f0-a952-4f58-adb2-8a86d074325a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:03.214244 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576d1f0-a952-4f58-adb2-8a86d074325a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:03.251597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 197/370] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:03.251801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576d1f0-a952-4f58-adb2-8a86d074325a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:03.276268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:03.331039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:03.341491 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0dd82-7232-431e-ab9c-214073591a30 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:03.414356 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb33c46-991e-4ef0-9b48-96a8aae7b070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:03.417184 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb33c46-991e-4ef0-9b48-96a8aae7b070 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:03.418101 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb33c46-991e-4ef0-9b48-96a8aae7b070 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:03.422604 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb33c46-991e-4ef0-9b48-96a8aae7b070 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:03.422604 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb33c46-991e-4ef0-9b48-96a8aae7b070 None admin] RBAC: Adding query filter params (role.id=8655041bde1948a1be442c6239496c22, user.id=89cad0e7021f447dbb7268c90cfbcdb9, scope.system=all) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:03.422604 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb33c46-991e-4ef0-9b48-96a8aae7b070 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:03.422604 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb33c46-991e-4ef0-9b48-96a8aae7b070 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:03.455078 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 174/371] 10.222.0.67 () {56 vars in 1308 bytes} [Tue May 31 14:25:03 2022] GET /identity/v3/role_assignments?user.id=89cad0e7021f447dbb7268c90cfbcdb9&scope.system=all&role.id=8655041bde1948a1be442c6239496c22 => generated 510 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:03.456976 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb33c46-991e-4ef0-9b48-96a8aae7b070 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:03.504909 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576d1f0-a952-4f58-adb2-8a86d074325a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:03.592189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d6393-959e-45a3-8520-86b1f31568fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:03.594688 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d6393-959e-45a3-8520-86b1f31568fa None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:03.594688 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d6393-959e-45a3-8520-86b1f31568fa None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:03.594688 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d6393-959e-45a3-8520-86b1f31568fa None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:03.594688 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d6393-959e-45a3-8520-86b1f31568fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:03.594688 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d6393-959e-45a3-8520-86b1f31568fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:03.601454 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d6393-959e-45a3-8520-86b1f31568fa None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:03.635066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 198/372] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:25:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:03.639804 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d6393-959e-45a3-8520-86b1f31568fa None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:03.703886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb33c46-991e-4ef0-9b48-96a8aae7b070 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:03.770234 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0fac9-722b-46b7-9c17-90bf22f6af80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:03.774447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0fac9-722b-46b7-9c17-90bf22f6af80 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:03.775941 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0fac9-722b-46b7-9c17-90bf22f6af80 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:03.778630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0fac9-722b-46b7-9c17-90bf22f6af80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:03.783784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:03.831015 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:03.863330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d6393-959e-45a3-8520-86b1f31568fa None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:03.911213 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0fac9-722b-46b7-9c17-90bf22f6af80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:03.912690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0fac9-722b-46b7-9c17-90bf22f6af80 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:03.931667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e132aa-c61e-45e1-b1bd-3a43f65c00e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:03.932353 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e132aa-c61e-45e1-b1bd-3a43f65c00e5 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:03.932614 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e132aa-c61e-45e1-b1bd-3a43f65c00e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:03.932895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e132aa-c61e-45e1-b1bd-3a43f65c00e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:04.044810 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e132aa-c61e-45e1-b1bd-3a43f65c00e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:04.047342 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e132aa-c61e-45e1-b1bd-3a43f65c00e5 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:04.105743 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 175/373] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:25:04.111675 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0fac9-722b-46b7-9c17-90bf22f6af80 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:04.165782 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:04.228786 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:04.266501 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 199/374] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:25:04.269150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e132aa-c61e-45e1-b1bd-3a43f65c00e5 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:04.309729 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0fac9-722b-46b7-9c17-90bf22f6af80 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:04.390233 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fad5a1-f751-475c-96f9-1f994be0a338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:04.391141 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fad5a1-f751-475c-96f9-1f994be0a338 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:04.391424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fad5a1-f751-475c-96f9-1f994be0a338 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:04.391698 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fad5a1-f751-475c-96f9-1f994be0a338 None admin] PATH_INFO: `/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:04.394307 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fad5a1-f751-475c-96f9-1f994be0a338 None admin] RBAC: Authorizing `identity:get_project(project_id=bef8d3a846e34a8daf7eecb46a5fd2f8)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:04.394440 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e132aa-c61e-45e1-b1bd-3a43f65c00e5 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:04.395784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fad5a1-f751-475c-96f9-1f994be0a338 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:04.398477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 176/375] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:25:04 2022] GET /identity/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8 => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:25:04.403600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fad5a1-f751-475c-96f9-1f994be0a338 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:04.432388 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673572d-331a-4c99-863f-22ce5dde2857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:04.433253 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673572d-331a-4c99-863f-22ce5dde2857 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:04.433537 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673572d-331a-4c99-863f-22ce5dde2857 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:04.433812 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673572d-331a-4c99-863f-22ce5dde2857 None admin] PATH_INFO: `/v3/projects/929ceca16868457ca23cb37daf0ee36d` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:04.435081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673572d-331a-4c99-863f-22ce5dde2857 None admin] RBAC: Authorizing `identity:get_project(project_id=929ceca16868457ca23cb37daf0ee36d)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:04.436549 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673572d-331a-4c99-863f-22ce5dde2857 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:04.438739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 200/376] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:25:04 2022] GET /identity/v3/projects/929ceca16868457ca23cb37daf0ee36d => generated 344 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:04.444828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673572d-331a-4c99-863f-22ce5dde2857 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:04.548285 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:04.609585 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:04.616446 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fad5a1-f751-475c-96f9-1f994be0a338 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:04.646795 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673572d-331a-4c99-863f-22ce5dde2857 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:04.686044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82cc4e-d9fa-414b-a00d-931ceb6d434c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:04.687631 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82cc4e-d9fa-414b-a00d-931ceb6d434c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:04.687924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82cc4e-d9fa-414b-a00d-931ceb6d434c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:04.688211 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82cc4e-d9fa-414b-a00d-931ceb6d434c None admin] PATH_INFO: `/v3/groups/93ca2d3407df4b40a423023aec0efbbe` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:04.690055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82cc4e-d9fa-414b-a00d-931ceb6d434c None admin] RBAC: Authorizing `identity:get_group(group_id=93ca2d3407df4b40a423023aec0efbbe)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:04.692064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82cc4e-d9fa-414b-a00d-931ceb6d434c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:04.694534 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 177/377] 10.222.0.67 () {56 vars in 1180 bytes} [Tue May 31 14:25:04 2022] GET /identity/v3/groups/93ca2d3407df4b40a423023aec0efbbe => generated 226 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:25:04.695671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9c897-382f-453b-a8db-44ff6874f6a3 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:04.695890 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9c897-382f-453b-a8db-44ff6874f6a3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:04.696187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9c897-382f-453b-a8db-44ff6874f6a3 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:04.697678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 178/378] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:04.698911 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a1192-bbc0-46b6-8dcb-4ef39856a6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:04.699890 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a1192-bbc0-46b6-8dcb-4ef39856a6f3 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:04.700244 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a1192-bbc0-46b6-8dcb-4ef39856a6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:04.700594 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a1192-bbc0-46b6-8dcb-4ef39856a6f3 None admin] PATH_INFO: `/v3/groups/dcc5a08271114a7abb972516ac9472fc` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:04.703085 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae73d3-9482-40cf-b18b-6dc3e67b1857 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:04.703659 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae73d3-9482-40cf-b18b-6dc3e67b1857 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:04.703947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae73d3-9482-40cf-b18b-6dc3e67b1857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:04.707577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a1192-bbc0-46b6-8dcb-4ef39856a6f3 None admin] RBAC: Authorizing `identity:get_group(group_id=dcc5a08271114a7abb972516ac9472fc)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:04.709034 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a1192-bbc0-46b6-8dcb-4ef39856a6f3 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:04.710954 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 201/379] 10.222.0.67 () {56 vars in 1180 bytes} [Tue May 31 14:25:04 2022] GET /identity/v3/groups/dcc5a08271114a7abb972516ac9472fc => generated 229 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:04.711629 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a1192-bbc0-46b6-8dcb-4ef39856a6f3 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:04.724339 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-beae73d3-9482-40cf-b18b-6dc3e67b1857 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:04.726098 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-beae73d3-9482-40cf-b18b-6dc3e67b1857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:04.820295 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 179/380] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 31 14:25:04.821251 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae73d3-9482-40cf-b18b-6dc3e67b1857 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:04.852081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a1192-bbc0-46b6-8dcb-4ef39856a6f3 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:04.872667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:04.892347 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b2240-12ac-4141-8aed-c3fa7a96bd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:04.893121 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b2240-12ac-4141-8aed-c3fa7a96bd8a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:04.893410 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b2240-12ac-4141-8aed-c3fa7a96bd8a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:04.893688 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b2240-12ac-4141-8aed-c3fa7a96bd8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:04.894558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9b2240-12ac-4141-8aed-c3fa7a96bd8a None admin] RBAC: Adding query filter params (group.id=93ca2d3407df4b40a423023aec0efbbe, role.id=1b9c18d4b8b540bf9825080748cec3bc, scope.project.id=bef8d3a846e34a8daf7eecb46a5fd2f8) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:04.894884 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9b2240-12ac-4141-8aed-c3fa7a96bd8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:04.896583 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9b2240-12ac-4141-8aed-c3fa7a96bd8a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:04.903600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 202/381] 10.222.0.67 () {56 vars in 1376 bytes} [Tue May 31 14:25:04 2022] GET /identity/v3/role_assignments?group.id=93ca2d3407df4b40a423023aec0efbbe&scope.project.id=bef8d3a846e34a8daf7eecb46a5fd2f8&role.id=1b9c18d4b8b540bf9825080748cec3bc => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:04.904505 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b2240-12ac-4141-8aed-c3fa7a96bd8a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:04.957109 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:04.966976 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae73d3-9482-40cf-b18b-6dc3e67b1857 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:05.009560 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ad538-5222-4e26-8ae6-0243693fef89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:05.010712 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ad538-5222-4e26-8ae6-0243693fef89 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:05.011205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ad538-5222-4e26-8ae6-0243693fef89 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:05.011709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ad538-5222-4e26-8ae6-0243693fef89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:05.013418 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ad538-5222-4e26-8ae6-0243693fef89 None admin] RBAC: Adding query filter params (group.id=dcc5a08271114a7abb972516ac9472fc, role.id=e8d679596ffb45939229d1e2a13ce9b0, scope.project.id=929ceca16868457ca23cb37daf0ee36d) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:05.015357 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ad538-5222-4e26-8ae6-0243693fef89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:05.017285 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ad538-5222-4e26-8ae6-0243693fef89 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:05.020266 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b2240-12ac-4141-8aed-c3fa7a96bd8a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:05.024887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 180/382] 10.222.0.67 () {56 vars in 1376 bytes} [Tue May 31 14:25:04 2022] GET /identity/v3/role_assignments?group.id=dcc5a08271114a7abb972516ac9472fc&scope.project.id=929ceca16868457ca23cb37daf0ee36d&role.id=e8d679596ffb45939229d1e2a13ce9b0 => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:05.060240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe6292f-e7c4-4ed4-a9b7-396f35c91edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:05.061104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6292f-e7c4-4ed4-a9b7-396f35c91edf None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:05.061389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6292f-e7c4-4ed4-a9b7-396f35c91edf None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:05.061667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6292f-e7c4-4ed4-a9b7-396f35c91edf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:05.074557 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe6292f-e7c4-4ed4-a9b7-396f35c91edf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:05.084057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-0fe6292f-e7c4-4ed4-a9b7-396f35c91edf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:25:05.084057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:05.084057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:05.084057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:05.084057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:05.084057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:05.084057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:05.084057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:05.084057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:05.084057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:05.084057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:05.084057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:05.084057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:05.084057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:05.084057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:05.084057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:05.085028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:05.085028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:05.085028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:05.085028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:05.085028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:05.085028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:05.085028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:05.085028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:05.085028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:05.085028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:05.085028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:05.085028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:05.085028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:05.085028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:05.085028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:05.085028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:05.085835 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:05.085835 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:05.085835 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:05.085835 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:05.085835 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:05.085835 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:05.085835 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:05.085835 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:05.085835 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:05.085835 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:05.085835 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:25:05.085835 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:05.085835 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 203/383] 10.222.0.67 () {56 vars in 1114 bytes} [Tue May 31 14:25:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:05.093910 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ad538-5222-4e26-8ae6-0243693fef89 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:05.228758 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ad538-5222-4e26-8ae6-0243693fef89 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:05.267499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84884f0-ef10-4ed0-a31e-aad44490af99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:05.268202 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84884f0-ef10-4ed0-a31e-aad44490af99 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:05.268490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84884f0-ef10-4ed0-a31e-aad44490af99 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:05.268771 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84884f0-ef10-4ed0-a31e-aad44490af99 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:05.269547 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84884f0-ef10-4ed0-a31e-aad44490af99 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:05.269857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84884f0-ef10-4ed0-a31e-aad44490af99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:05.271305 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84884f0-ef10-4ed0-a31e-aad44490af99 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:05.282393 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 181/384] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:05.288710 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe6292f-e7c4-4ed4-a9b7-396f35c91edf None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:05.356539 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:05.431515 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe6292f-e7c4-4ed4-a9b7-396f35c91edf None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:05.468569 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9d079-4611-43f8-964d-ca8495107097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:05.469755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9d079-4611-43f8-964d-ca8495107097 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:05.470236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9d079-4611-43f8-964d-ca8495107097 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:05.470733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9d079-4611-43f8-964d-ca8495107097 None admin] PATH_INFO: `/v3/groups/93ca2d3407df4b40a423023aec0efbbe` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:05.472150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9d079-4611-43f8-964d-ca8495107097 None admin] RBAC: Authorizing `identity:get_group(group_id=93ca2d3407df4b40a423023aec0efbbe)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:05.473796 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9d079-4611-43f8-964d-ca8495107097 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:05.475978 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 204/385] 10.222.0.67 () {56 vars in 1180 bytes} [Tue May 31 14:25:05 2022] GET /identity/v3/groups/93ca2d3407df4b40a423023aec0efbbe => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:05.482173 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84884f0-ef10-4ed0-a31e-aad44490af99 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:05.511050 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:05.598742 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84884f0-ef10-4ed0-a31e-aad44490af99 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:05.640594 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2832f5e-62d1-4a29-834a-130d6e284caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:05.641484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2832f5e-62d1-4a29-834a-130d6e284caa None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:05.641775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2832f5e-62d1-4a29-834a-130d6e284caa None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:05.642050 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2832f5e-62d1-4a29-834a-130d6e284caa None admin] PATH_INFO: `/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:05.643398 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2832f5e-62d1-4a29-834a-130d6e284caa None admin] RBAC: Authorizing `identity:get_project(project_id=e3f26f512b354f6ebdbf6c8d3cc3e472)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:05.644855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2832f5e-62d1-4a29-834a-130d6e284caa None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:05.646766 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 182/386] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:25:05 2022] GET /identity/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:05.654309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9d079-4611-43f8-964d-ca8495107097 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:05.692737 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803e35-8d48-4b0b-9b35-71efccf96820 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:05.693175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803e35-8d48-4b0b-9b35-71efccf96820 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:05.693805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803e35-8d48-4b0b-9b35-71efccf96820 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:05.696202 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 183/387] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:05.702903 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ec249-3ef4-46eb-8514-1d429a7c3c60 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:05.703248 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ec249-3ef4-46eb-8514-1d429a7c3c60 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:05.703588 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ec249-3ef4-46eb-8514-1d429a7c3c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:05.728047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d71ec249-3ef4-46eb-8514-1d429a7c3c60 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:05.730284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d71ec249-3ef4-46eb-8514-1d429a7c3c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:05.790541 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9d079-4611-43f8-964d-ca8495107097 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:05.801050 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:05.807474 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 184/388] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:05.831351 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ec249-3ef4-46eb-8514-1d429a7c3c60 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:05.835270 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e5f92-e1ee-4b67-a7f3-640f2482b8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:05.836171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e5f92-e1ee-4b67-a7f3-640f2482b8b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:05.836450 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e5f92-e1ee-4b67-a7f3-640f2482b8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:05.836723 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e5f92-e1ee-4b67-a7f3-640f2482b8b1 None admin] PATH_INFO: `/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472/groups/93ca2d3407df4b40a423023aec0efbbe/roles/1b9c18d4b8b540bf9825080748cec3bc` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:05.839774 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e5f92-e1ee-4b67-a7f3-640f2482b8b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=e3f26f512b354f6ebdbf6c8d3cc3e472, group_id=93ca2d3407df4b40a423023aec0efbbe, role_id=1b9c18d4b8b540bf9825080748cec3bc)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:05.841928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e5f92-e1ee-4b67-a7f3-640f2482b8b1 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:05.858823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:05.862115 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 205/389] 10.222.0.67 () {58 vars in 1442 bytes} [Tue May 31 14:25:05 2022] PUT /identity/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472/groups/93ca2d3407df4b40a423023aec0efbbe/roles/1b9c18d4b8b540bf9825080748cec3bc => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:25:05.966463 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ec249-3ef4-46eb-8514-1d429a7c3c60 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:06.015676 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fd672-01d3-4479-be73-72da2272991d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:06.016508 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fd672-01d3-4479-be73-72da2272991d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:06.016798 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fd672-01d3-4479-be73-72da2272991d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:06.017079 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fd672-01d3-4479-be73-72da2272991d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:06.044115 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4fd672-01d3-4479-be73-72da2272991d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:06.052205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-0d4fd672-01d3-4479-be73-72da2272991d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 31 14:25:06.052205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:06.052205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:06.052205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:06.052205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:06.052205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:06.052205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:06.052205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:06.052205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:06.052205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:06.052205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:06.052205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:06.052205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:06.052205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:06.052205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:06.052205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:06.053192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:06.053192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:06.053192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:06.053192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:06.053192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:06.053192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:06.053192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:06.053192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:06.053192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:06.053192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:06.053192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:06.053192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:06.053192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:06.053192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:06.053192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:06.053192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:06.053975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:06.053975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:06.053975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:06.053975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:06.053975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:06.053975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:06.053975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:06.053975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:06.053975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:06.053975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:06.053975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 31 14:25:06.053975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:06.056082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 185/390] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:25:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:25:06.061583 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e5f92-e1ee-4b67-a7f3-640f2482b8b1 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:06.198374 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e5f92-e1ee-4b67-a7f3-640f2482b8b1 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:06.235931 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f046ae8-6e62-467a-b233-6c1f53f8d9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:06.236541 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f046ae8-6e62-467a-b233-6c1f53f8d9f5 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:06.236827 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f046ae8-6e62-467a-b233-6c1f53f8d9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:06.237107 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f046ae8-6e62-467a-b233-6c1f53f8d9f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:06.237863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f046ae8-6e62-467a-b233-6c1f53f8d9f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:06.238174 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f046ae8-6e62-467a-b233-6c1f53f8d9f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:06.239610 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f046ae8-6e62-467a-b233-6c1f53f8d9f5 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:06.248341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 206/391] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:25:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:06.253829 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fd672-01d3-4479-be73-72da2272991d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:06.346326 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:06.374562 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fd672-01d3-4479-be73-72da2272991d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:06.424563 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53b6b4-dfb1-4aeb-bbee-7f95770484de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:06.425433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53b6b4-dfb1-4aeb-bbee-7f95770484de None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:06.425717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53b6b4-dfb1-4aeb-bbee-7f95770484de None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:06.425991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53b6b4-dfb1-4aeb-bbee-7f95770484de None admin] PATH_INFO: `/v3/groups/93ca2d3407df4b40a423023aec0efbbe` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:06.431367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53b6b4-dfb1-4aeb-bbee-7f95770484de None admin] RBAC: Authorizing `identity:get_group(group_id=93ca2d3407df4b40a423023aec0efbbe)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:06.432889 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53b6b4-dfb1-4aeb-bbee-7f95770484de None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:06.434732 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 186/392] 10.222.0.67 () {56 vars in 1180 bytes} [Tue May 31 14:25:06 2022] GET /identity/v3/groups/93ca2d3407df4b40a423023aec0efbbe => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:25:06.440324 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f046ae8-6e62-467a-b233-6c1f53f8d9f5 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:06.484064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:06.587690 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f046ae8-6e62-467a-b233-6c1f53f8d9f5 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:06.615092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9759e8-8d3b-4c84-b660-c8793980c2f7 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:06.615397 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9759e8-8d3b-4c84-b660-c8793980c2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:06.615658 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9759e8-8d3b-4c84-b660-c8793980c2f7 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:06.618076 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 187/393] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:06.622726 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210feae-e9aa-4c57-8f8e-21be818bbc2c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:06.623010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210feae-e9aa-4c57-8f8e-21be818bbc2c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:06.623290 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210feae-e9aa-4c57-8f8e-21be818bbc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:06.641567 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b00181-b845-4b10-a21e-7865b6c79ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:06.642840 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b00181-b845-4b10-a21e-7865b6c79ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:06.643320 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b00181-b845-4b10-a21e-7865b6c79ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:06.643797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b00181-b845-4b10-a21e-7865b6c79ff9 None admin] PATH_INFO: `/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:06.646282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b00181-b845-4b10-a21e-7865b6c79ff9 None admin] RBAC: Authorizing `identity:get_project(project_id=e3f26f512b354f6ebdbf6c8d3cc3e472)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:06.647985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b00181-b845-4b10-a21e-7865b6c79ff9 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:06.649884 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 207/394] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:25:06 2022] GET /identity/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:06.654139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2210feae-e9aa-4c57-8f8e-21be818bbc2c None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:06.655437 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b00181-b845-4b10-a21e-7865b6c79ff9 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:06.665524 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2210feae-e9aa-4c57-8f8e-21be818bbc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:06.802607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 188/395] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:06.823783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:06.830531 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b00181-b845-4b10-a21e-7865b6c79ff9 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:06.836704 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210feae-e9aa-4c57-8f8e-21be818bbc2c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:06.901902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd08327-7363-4877-8c92-a4f96716cc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:06.902982 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd08327-7363-4877-8c92-a4f96716cc53 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:06.907245 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd08327-7363-4877-8c92-a4f96716cc53 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:06.907544 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd08327-7363-4877-8c92-a4f96716cc53 None admin] PATH_INFO: `/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472/groups/93ca2d3407df4b40a423023aec0efbbe/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:06.909646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd08327-7363-4877-8c92-a4f96716cc53 None admin] RBAC: Authorizing `identity:create_grant(project_id=e3f26f512b354f6ebdbf6c8d3cc3e472, group_id=93ca2d3407df4b40a423023aec0efbbe, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:06.911400 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd08327-7363-4877-8c92-a4f96716cc53 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:06.953782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 208/396] 10.222.0.67 () {58 vars in 1442 bytes} [Tue May 31 14:25:06 2022] PUT /identity/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472/groups/93ca2d3407df4b40a423023aec0efbbe/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:25:06.954050 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:06.962887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210feae-e9aa-4c57-8f8e-21be818bbc2c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:07.005197 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef65d1-a525-4771-8cb4-a9f3fafca7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:07.006364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef65d1-a525-4771-8cb4-a9f3fafca7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:07.006803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef65d1-a525-4771-8cb4-a9f3fafca7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:07.007212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef65d1-a525-4771-8cb4-a9f3fafca7a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:07.022335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef65d1-a525-4771-8cb4-a9f3fafca7a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:07.032952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-80ef65d1-a525-4771-8cb4-a9f3fafca7a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 31 14:25:07.032952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:07.032952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:07.032952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:07.032952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:07.032952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:07.032952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:07.032952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:07.032952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:07.032952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:07.032952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:07.032952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:07.032952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:07.032952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:07.032952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:07.032952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:07.033965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:07.033965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:07.033965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:07.033965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:07.033965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:07.033965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:07.033965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:07.033965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:07.033965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:07.033965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:07.033965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:07.033965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:07.033965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:07.033965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:07.033965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:07.033965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:07.035010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:07.035010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:07.035010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:07.035010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:07.035010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:07.035010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:07.035010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:07.035010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:07.035010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:07.035010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:07.035010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 31 14:25:07.035010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:07.035010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 189/397] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:25:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:25:07.044290 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd08327-7363-4877-8c92-a4f96716cc53 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:07.229067 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd08327-7363-4877-8c92-a4f96716cc53 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:07.265192 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44867f-2c4a-4845-a4cc-b3b5e6d5ea4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:07.265877 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44867f-2c4a-4845-a4cc-b3b5e6d5ea4a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:07.266162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44867f-2c4a-4845-a4cc-b3b5e6d5ea4a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:07.266441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44867f-2c4a-4845-a4cc-b3b5e6d5ea4a None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:07.267246 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e44867f-2c4a-4845-a4cc-b3b5e6d5ea4a None admin] RBAC: Adding query filter params (name=member) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:07.267622 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e44867f-2c4a-4845-a4cc-b3b5e6d5ea4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:07.269016 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e44867f-2c4a-4845-a4cc-b3b5e6d5ea4a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:07.278312 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 209/398] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:25:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:07.286718 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef65d1-a525-4771-8cb4-a9f3fafca7a3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:07.444458 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef65d1-a525-4771-8cb4-a9f3fafca7a3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:07.491530 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe93272-2991-473b-aec4-e127a8a53a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:07.492334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe93272-2991-473b-aec4-e127a8a53a4a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:07.492626 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe93272-2991-473b-aec4-e127a8a53a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:07.492911 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe93272-2991-473b-aec4-e127a8a53a4a None admin] PATH_INFO: `/v3/groups/93ca2d3407df4b40a423023aec0efbbe` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:07.494131 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe93272-2991-473b-aec4-e127a8a53a4a None admin] RBAC: Authorizing `identity:get_group(group_id=93ca2d3407df4b40a423023aec0efbbe)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:07.495652 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe93272-2991-473b-aec4-e127a8a53a4a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:07.497918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 190/399] 10.222.0.67 () {56 vars in 1180 bytes} [Tue May 31 14:25:07 2022] GET /identity/v3/groups/93ca2d3407df4b40a423023aec0efbbe => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:07.503701 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44867f-2c4a-4845-a4cc-b3b5e6d5ea4a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:07.537745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:07.643753 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44867f-2c4a-4845-a4cc-b3b5e6d5ea4a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:07.688516 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a48853-16a3-4f09-b3e8-3f0b0403aa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:07.689385 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a48853-16a3-4f09-b3e8-3f0b0403aa4a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:07.689668 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a48853-16a3-4f09-b3e8-3f0b0403aa4a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:07.689942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a48853-16a3-4f09-b3e8-3f0b0403aa4a None admin] PATH_INFO: `/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:07.695368 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a48853-16a3-4f09-b3e8-3f0b0403aa4a None admin] RBAC: Authorizing `identity:get_project(project_id=bef8d3a846e34a8daf7eecb46a5fd2f8)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:07.696950 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a48853-16a3-4f09-b3e8-3f0b0403aa4a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:07.699020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 210/400] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:25:07 2022] GET /identity/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:07.705113 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe93272-2991-473b-aec4-e127a8a53a4a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:07.720106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:07.836458 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe93272-2991-473b-aec4-e127a8a53a4a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:07.883369 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd00c7d-31d6-488c-a699-80ffb37fabb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:07.884264 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd00c7d-31d6-488c-a699-80ffb37fabb9 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:07.884553 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd00c7d-31d6-488c-a699-80ffb37fabb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:07.884831 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd00c7d-31d6-488c-a699-80ffb37fabb9 None admin] PATH_INFO: `/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8/groups/93ca2d3407df4b40a423023aec0efbbe/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:07.886848 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd00c7d-31d6-488c-a699-80ffb37fabb9 None admin] RBAC: Authorizing `identity:create_grant(project_id=bef8d3a846e34a8daf7eecb46a5fd2f8, group_id=93ca2d3407df4b40a423023aec0efbbe, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:07.888344 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd00c7d-31d6-488c-a699-80ffb37fabb9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:07.927228 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 191/401] 10.222.0.67 () {58 vars in 1442 bytes} [Tue May 31 14:25:07 2022] PUT /identity/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8/groups/93ca2d3407df4b40a423023aec0efbbe/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:25:08.102905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6426b-975e-4b8c-8459-f92cdce2e508 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:08.103280 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6426b-975e-4b8c-8459-f92cdce2e508 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:08.103497 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6426b-975e-4b8c-8459-f92cdce2e508 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:08.105203 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 211/402] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:08.116475 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0c4a3-4c39-4b41-ba15-7e14d11d7786 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:08.122176 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0c4a3-4c39-4b41-ba15-7e14d11d7786 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:08.122477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0c4a3-4c39-4b41-ba15-7e14d11d7786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:08.155549 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1cb0c4a3-4c39-4b41-ba15-7e14d11d7786 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:08.157095 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1cb0c4a3-4c39-4b41-ba15-7e14d11d7786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:08.243219 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:08.253831 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 192/403] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:08.276587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c6426b-975e-4b8c-8459-f92cdce2e508 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:08.394898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:08.415302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c6426b-975e-4b8c-8459-f92cdce2e508 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:08.465134 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a013f30-0dda-4bfc-8b33-23872ab213c4 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:08.465580 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a013f30-0dda-4bfc-8b33-23872ab213c4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:08.466493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a013f30-0dda-4bfc-8b33-23872ab213c4 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:08.472229 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 193/404] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:08 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:08.476808 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202bd55c-9820-4aba-8f2f-f920fd87e4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:08.477636 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202bd55c-9820-4aba-8f2f-f920fd87e4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:08.477923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202bd55c-9820-4aba-8f2f-f920fd87e4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:08.478199 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202bd55c-9820-4aba-8f2f-f920fd87e4f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:08.481631 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e5198-9e43-455e-9c87-f2a724dda4f1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:08.481925 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e5198-9e43-455e-9c87-f2a724dda4f1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:08.482209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e5198-9e43-455e-9c87-f2a724dda4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:08.494743 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202bd55c-9820-4aba-8f2f-f920fd87e4f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:08.503510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-202bd55c-9820-4aba-8f2f-f920fd87e4f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:25:08.503510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:08.503510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:08.503510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:08.503510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:08.503510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:08.503510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:08.503510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:08.503510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:08.503510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:08.503510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:08.503510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:08.503510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:08.503510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:08.503510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:08.503510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:08.504918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:08.504918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:08.504918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:08.504918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:08.504918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:08.504918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:08.504918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:08.504918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:08.504918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:08.504918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:08.504918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:08.504918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:08.504918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:08.504918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:08.504918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:08.504918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:08.506168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:08.506168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:08.506168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:08.506168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:08.506168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:08.506168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:08.506168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:08.506168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:08.506168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:08.506168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:08.506168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:25:08.506168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:08.514895 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ad0e5198-9e43-455e-9c87-f2a724dda4f1 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:08.516085 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ad0e5198-9e43-455e-9c87-f2a724dda4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:08.518399 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 212/405] 10.222.0.67 () {56 vars in 1114 bytes} [Tue May 31 14:25:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:08.521070 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202bd55c-9820-4aba-8f2f-f920fd87e4f4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:08.638770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 194/406] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:08.640475 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3fb5a-62e0-4514-bf6c-af900d8016af None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:08.640941 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3fb5a-62e0-4514-bf6c-af900d8016af None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:08.641442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3fb5a-62e0-4514-bf6c-af900d8016af None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:08.645074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 195/407] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:08.651817 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9b948-c09b-4c58-846f-e5248fac8d2a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:08.652321 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9b948-c09b-4c58-846f-e5248fac8d2a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:08.652813 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9b948-c09b-4c58-846f-e5248fac8d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:08.677819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f5d9b948-c09b-4c58-846f-e5248fac8d2a None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:08.679347 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f5d9b948-c09b-4c58-846f-e5248fac8d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:08.690209 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202bd55c-9820-4aba-8f2f-f920fd87e4f4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:08.726550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccea0ae-ac02-4f02-9631-853a73272e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:08.727474 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccea0ae-ac02-4f02-9631-853a73272e3b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:08.727911 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccea0ae-ac02-4f02-9631-853a73272e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:08.728364 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccea0ae-ac02-4f02-9631-853a73272e3b None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:08.729257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccea0ae-ac02-4f02-9631-853a73272e3b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:08.729713 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccea0ae-ac02-4f02-9631-853a73272e3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:08.731863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccea0ae-ac02-4f02-9631-853a73272e3b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:08.740939 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 213/408] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:08.748578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccea0ae-ac02-4f02-9631-853a73272e3b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:08.764000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:08.770736 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 196/409] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:08.771846 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9b948-c09b-4c58-846f-e5248fac8d2a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:08.862984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:08.899956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccea0ae-ac02-4f02-9631-853a73272e3b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:08.918379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9b948-c09b-4c58-846f-e5248fac8d2a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:08.938295 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dcbc7b-f9bf-4e6c-a1f9-f9380cd51eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:08.939488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcbc7b-f9bf-4e6c-a1f9-f9380cd51eef None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:08.940059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcbc7b-f9bf-4e6c-a1f9-f9380cd51eef None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:08.940534 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcbc7b-f9bf-4e6c-a1f9-f9380cd51eef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:08.952576 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dcbc7b-f9bf-4e6c-a1f9-f9380cd51eef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:08.957030 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb6ffe-476b-4153-8071-7de2f01a7273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:08.958097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb6ffe-476b-4153-8071-7de2f01a7273 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:08.958529 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb6ffe-476b-4153-8071-7de2f01a7273 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:08.958983 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb6ffe-476b-4153-8071-7de2f01a7273 None admin] PATH_INFO: `/v3/groups/93ca2d3407df4b40a423023aec0efbbe` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:08.960341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb6ffe-476b-4153-8071-7de2f01a7273 None admin] RBAC: Authorizing `identity:get_group(group_id=93ca2d3407df4b40a423023aec0efbbe)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:08.961916 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb6ffe-476b-4153-8071-7de2f01a7273 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:08.962144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-e9dcbc7b-f9bf-4e6c-a1f9-f9380cd51eef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:25:08.962144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:08.962144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:08.962144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:08.962144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:08.962144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:08.962144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:08.962144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:08.962144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:08.962144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:08.962144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:08.962144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:08.962144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:08.962144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:08.962144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:08.962144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:08.963262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:08.963262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:08.963262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:08.963262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:08.963262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:08.963262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:08.963262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:08.963262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:08.963262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:08.963262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:08.963262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:08.963262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:08.963262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:08.963262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:08.963262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:08.963262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:08.964243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:08.964243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:08.964243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:08.964243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:08.964243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:08.964243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:08.964243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:08.964243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:08.964243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:08.964243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:08.964243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:25:08.964243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:08.966146 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 214/410] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:25:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:08.970677 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dcbc7b-f9bf-4e6c-a1f9-f9380cd51eef None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:08.972285 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 197/411] 10.222.0.67 () {56 vars in 1180 bytes} [Tue May 31 14:25:08 2022] GET /identity/v3/groups/93ca2d3407df4b40a423023aec0efbbe => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:08.973286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb6ffe-476b-4153-8071-7de2f01a7273 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:09.058615 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:09.100407 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb6ffe-476b-4153-8071-7de2f01a7273 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:09.102879 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dcbc7b-f9bf-4e6c-a1f9-f9380cd51eef None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:09.139695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:09.160977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9fa7de-528c-499c-91f5-9549add6b925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:09.161197 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc54e0-4477-4beb-a249-8300964a34f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:09.161764 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc54e0-4477-4beb-a249-8300964a34f0 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:09.162047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc54e0-4477-4beb-a249-8300964a34f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:09.162325 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc54e0-4477-4beb-a249-8300964a34f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:09.163542 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fa7de-528c-499c-91f5-9549add6b925 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:09.164565 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fa7de-528c-499c-91f5-9549add6b925 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:09.165751 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fa7de-528c-499c-91f5-9549add6b925 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:09.167251 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9fa7de-528c-499c-91f5-9549add6b925 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:09.168886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9fa7de-528c-499c-91f5-9549add6b925 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:09.172389 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9fa7de-528c-499c-91f5-9549add6b925 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:09.174409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc54e0-4477-4beb-a249-8300964a34f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:09.184282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-a8dc54e0-4477-4beb-a249-8300964a34f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:25:09.184282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:09.184282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:09.184282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:09.184282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:09.184282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:09.184282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:09.184282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:09.184282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:09.184282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:09.184282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:09.184282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:09.184282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:09.184282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:09.184282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:09.184282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:09.185745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:09.185745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:09.185745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:09.185745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:09.185745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:09.185745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:09.185745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:09.185745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:09.185745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:09.185745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:09.185745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:09.185745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:09.185745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:09.185745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:09.185745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:09.185745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:09.186711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:09.186711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:09.186711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:09.186711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:09.186711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:09.186711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:09.186711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:09.186711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:09.186711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:09.186711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:09.186711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:25:09.186711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:09.190420 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 198/412] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:25:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:09.192354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9fa7de-528c-499c-91f5-9549add6b925 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:09.194976 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 215/413] 10.222.0.67 () {56 vars in 1114 bytes} [Tue May 31 14:25:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:09.197195 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc54e0-4477-4beb-a249-8300964a34f0 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:09.331657 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc54e0-4477-4beb-a249-8300964a34f0 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:09.332541 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9fa7de-528c-499c-91f5-9549add6b925 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:09.371170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5f929-273e-492d-95f4-938fe89c5d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:09.372106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5f929-273e-492d-95f4-938fe89c5d69 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:09.372496 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5f929-273e-492d-95f4-938fe89c5d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:09.372682 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5f929-273e-492d-95f4-938fe89c5d69 None admin] PATH_INFO: `/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:09.373975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf5f929-273e-492d-95f4-938fe89c5d69 None admin] RBAC: Authorizing `identity:get_project(project_id=bef8d3a846e34a8daf7eecb46a5fd2f8)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:09.374993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:09.375915 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf5f929-273e-492d-95f4-938fe89c5d69 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:09.378069 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 199/414] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:25:09 2022] GET /identity/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:09.382723 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5f929-273e-492d-95f4-938fe89c5d69 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:09.389280 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ca93e-49e1-44c2-92bc-9264f01d674a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:09.390335 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ca93e-49e1-44c2-92bc-9264f01d674a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:09.390766 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ca93e-49e1-44c2-92bc-9264f01d674a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:09.391152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ca93e-49e1-44c2-92bc-9264f01d674a None admin] PATH_INFO: `/v3/groups/dcc5a08271114a7abb972516ac9472fc` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:09.392607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ca93e-49e1-44c2-92bc-9264f01d674a None admin] RBAC: Authorizing `identity:get_group(group_id=dcc5a08271114a7abb972516ac9472fc)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:09.394196 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ca93e-49e1-44c2-92bc-9264f01d674a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:09.400971 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 216/415] 10.222.0.67 () {56 vars in 1180 bytes} [Tue May 31 14:25:09 2022] GET /identity/v3/groups/dcc5a08271114a7abb972516ac9472fc => generated 229 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:09.401937 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ca93e-49e1-44c2-92bc-9264f01d674a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:09.463026 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:09.523789 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ca93e-49e1-44c2-92bc-9264f01d674a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:09.532304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5f929-273e-492d-95f4-938fe89c5d69 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:09.569330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33d27e-0e8b-471a-90bb-f8e2597be39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:09.570178 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33d27e-0e8b-471a-90bb-f8e2597be39a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:09.570555 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33d27e-0e8b-471a-90bb-f8e2597be39a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:09.570960 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33d27e-0e8b-471a-90bb-f8e2597be39a None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:09.571818 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33d27e-0e8b-471a-90bb-f8e2597be39a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:09.572253 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33d27e-0e8b-471a-90bb-f8e2597be39a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:09.573737 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33d27e-0e8b-471a-90bb-f8e2597be39a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:09.580193 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ecd1e-82be-4f5c-90d7-da81375b3777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:09.581965 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ecd1e-82be-4f5c-90d7-da81375b3777 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:09.582508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ecd1e-82be-4f5c-90d7-da81375b3777 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:09.582959 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ecd1e-82be-4f5c-90d7-da81375b3777 None admin] PATH_INFO: `/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8/groups/93ca2d3407df4b40a423023aec0efbbe/roles/1b9c18d4b8b540bf9825080748cec3bc` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:09.587462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ecd1e-82be-4f5c-90d7-da81375b3777 None admin] RBAC: Authorizing `identity:create_grant(project_id=bef8d3a846e34a8daf7eecb46a5fd2f8, group_id=93ca2d3407df4b40a423023aec0efbbe, role_id=1b9c18d4b8b540bf9825080748cec3bc)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:09.589055 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ecd1e-82be-4f5c-90d7-da81375b3777 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:09.589218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 200/416] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:09.590816 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33d27e-0e8b-471a-90bb-f8e2597be39a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:09.606867 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 217/417] 10.222.0.67 () {58 vars in 1442 bytes} [Tue May 31 14:25:09 2022] PUT /identity/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8/groups/93ca2d3407df4b40a423023aec0efbbe/roles/1b9c18d4b8b540bf9825080748cec3bc => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:25:09.612837 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ecd1e-82be-4f5c-90d7-da81375b3777 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:09.736799 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:09.757315 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33d27e-0e8b-471a-90bb-f8e2597be39a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:09.780055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:09.802261 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ecd1e-82be-4f5c-90d7-da81375b3777 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:09.803750 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a9e14-0066-4cf0-a9f3-5a2d9fd59b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:09.804965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a9e14-0066-4cf0-a9f3-5a2d9fd59b9a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:09.805329 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a9e14-0066-4cf0-a9f3-5a2d9fd59b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:09.805681 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a9e14-0066-4cf0-a9f3-5a2d9fd59b9a None admin] PATH_INFO: `/v3/projects/929ceca16868457ca23cb37daf0ee36d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:09.808094 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a9e14-0066-4cf0-a9f3-5a2d9fd59b9a None admin] RBAC: Authorizing `identity:get_project(project_id=929ceca16868457ca23cb37daf0ee36d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:09.810305 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a9e14-0066-4cf0-a9f3-5a2d9fd59b9a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:09.814286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 201/418] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:25:09 2022] GET /identity/v3/projects/929ceca16868457ca23cb37daf0ee36d => generated 344 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:09.817298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c4689-51f3-4d8b-9fe9-be51c454d718 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:09.818019 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c4689-51f3-4d8b-9fe9-be51c454d718 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:09.818964 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c4689-51f3-4d8b-9fe9-be51c454d718 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:09.826941 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 202/419] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:09.826941 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c4689-51f3-4d8b-9fe9-be51c454d718 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:09.853240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94482d-0def-4422-b9c9-2b9458e94aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:09.853898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94482d-0def-4422-b9c9-2b9458e94aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:09.854180 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94482d-0def-4422-b9c9-2b9458e94aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:09.854453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94482d-0def-4422-b9c9-2b9458e94aa7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:09.935033 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a94482d-0def-4422-b9c9-2b9458e94aa7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:09.936844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a94482d-0def-4422-b9c9-2b9458e94aa7 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:09.946946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c4689-51f3-4d8b-9fe9-be51c454d718 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:09.985872 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b7b50-366e-4898-b983-491aab483b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:09.986917 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b7b50-366e-4898-b983-491aab483b34 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:09.987144 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b7b50-366e-4898-b983-491aab483b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:09.987421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b7b50-366e-4898-b983-491aab483b34 None admin] PATH_INFO: `/v3/projects/929ceca16868457ca23cb37daf0ee36d/groups/dcc5a08271114a7abb972516ac9472fc/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:09.989554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b7b50-366e-4898-b983-491aab483b34 None admin] RBAC: Authorizing `identity:create_grant(project_id=929ceca16868457ca23cb37daf0ee36d, group_id=dcc5a08271114a7abb972516ac9472fc, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:09.991207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b7b50-366e-4898-b983-491aab483b34 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:10.001522 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:10.009338 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 203/420] 10.222.0.67 () {58 vars in 1442 bytes} [Tue May 31 14:25:09 2022] PUT /identity/v3/projects/929ceca16868457ca23cb37daf0ee36d/groups/dcc5a08271114a7abb972516ac9472fc/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:25:10.014708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9573e23-d470-4335-b9fc-85f6c80421b2 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:10.014708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9573e23-d470-4335-b9fc-85f6c80421b2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:10.014708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9573e23-d470-4335-b9fc-85f6c80421b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:10.032753 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e9573e23-d470-4335-b9fc-85f6c80421b2 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:10.034269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e9573e23-d470-4335-b9fc-85f6c80421b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:10.076120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 218/421] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:25:10.081890 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94482d-0def-4422-b9c9-2b9458e94aa7 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:10.084575 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:10.117653 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 204/422] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:10.144750 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9573e23-d470-4335-b9fc-85f6c80421b2 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:10.195383 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94482d-0def-4422-b9c9-2b9458e94aa7 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:10.234059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4760d-a31a-4fbe-aa1b-1776c7c6da31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:10.234706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4760d-a31a-4fbe-aa1b-1776c7c6da31 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:10.235006 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4760d-a31a-4fbe-aa1b-1776c7c6da31 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:10.235282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4760d-a31a-4fbe-aa1b-1776c7c6da31 None admin] PATH_INFO: `/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:10.236493 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4760d-a31a-4fbe-aa1b-1776c7c6da31 None admin] RBAC: Authorizing `identity:get_project(project_id=e3f26f512b354f6ebdbf6c8d3cc3e472)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:10.237995 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4760d-a31a-4fbe-aa1b-1776c7c6da31 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:10.239974 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 219/423] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:25:10 2022] GET /identity/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:25:10.245279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4760d-a31a-4fbe-aa1b-1776c7c6da31 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:10.262940 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9573e23-d470-4335-b9fc-85f6c80421b2 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:10.275439 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:10.329663 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0f241-cfc2-4ff6-b615-9c57312827e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:10.330826 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0f241-cfc2-4ff6-b615-9c57312827e9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:10.331257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0f241-cfc2-4ff6-b615-9c57312827e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:10.331676 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0f241-cfc2-4ff6-b615-9c57312827e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:10.351993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0f241-cfc2-4ff6-b615-9c57312827e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:10.360484 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4760d-a31a-4fbe-aa1b-1776c7c6da31 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:10.361033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-02f0f241-cfc2-4ff6-b615-9c57312827e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 31 14:25:10.361033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:10.361033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:10.361033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:10.361033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:10.361033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:10.361033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:10.361033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:10.361033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:10.361033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:10.361033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:10.361033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:10.361033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:10.361033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:10.361033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:10.361033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:10.362082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:10.362082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:10.362082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:10.362082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:10.362082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:10.362082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:10.362082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:10.362082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:10.362082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:10.362082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:10.362082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:10.362082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:10.362082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:10.362082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:10.362082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:10.362082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:10.362995 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:10.362995 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:10.362995 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:10.362995 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:10.362995 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:10.362995 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:10.362995 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:10.362995 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:10.362995 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:10.362995 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:10.362995 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 31 14:25:10.362995 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:10.365483 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 205/424] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:25:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:10.371299 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0f241-cfc2-4ff6-b615-9c57312827e9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:10.402426 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03061eae-b6b2-4b2a-8883-130ecda90de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:10.404185 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03061eae-b6b2-4b2a-8883-130ecda90de7 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:10.404481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03061eae-b6b2-4b2a-8883-130ecda90de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:10.404760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03061eae-b6b2-4b2a-8883-130ecda90de7 None admin] PATH_INFO: `/v3/groups/93ca2d3407df4b40a423023aec0efbbe` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:10.407205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03061eae-b6b2-4b2a-8883-130ecda90de7 None admin] RBAC: Authorizing `identity:get_group(group_id=93ca2d3407df4b40a423023aec0efbbe)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:10.409227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03061eae-b6b2-4b2a-8883-130ecda90de7 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:10.411525 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 220/425] 10.222.0.67 () {56 vars in 1180 bytes} [Tue May 31 14:25:10 2022] GET /identity/v3/groups/93ca2d3407df4b40a423023aec0efbbe => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:25:10.412245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:10.418879 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03061eae-b6b2-4b2a-8883-130ecda90de7 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:10.519851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0f241-cfc2-4ff6-b615-9c57312827e9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:10.539065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:10.543233 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03061eae-b6b2-4b2a-8883-130ecda90de7 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:10.582999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20d51c-883b-4999-a79a-3144e1490818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:10.583666 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20d51c-883b-4999-a79a-3144e1490818 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:10.583956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20d51c-883b-4999-a79a-3144e1490818 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:10.584237 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20d51c-883b-4999-a79a-3144e1490818 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:10.585083 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd20d51c-883b-4999-a79a-3144e1490818 None admin] RBAC: Adding query filter params (group.id=93ca2d3407df4b40a423023aec0efbbe, role.id=1b9c18d4b8b540bf9825080748cec3bc, scope.project.id=e3f26f512b354f6ebdbf6c8d3cc3e472) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:10.585400 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd20d51c-883b-4999-a79a-3144e1490818 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:10.586888 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd20d51c-883b-4999-a79a-3144e1490818 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:10.594511 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 221/426] 10.222.0.67 () {56 vars in 1376 bytes} [Tue May 31 14:25:10 2022] GET /identity/v3/role_assignments?group.id=93ca2d3407df4b40a423023aec0efbbe&scope.project.id=e3f26f512b354f6ebdbf6c8d3cc3e472&role.id=1b9c18d4b8b540bf9825080748cec3bc => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:25:10.640920 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0aee01-ee8c-4b3c-8b5a-c3db9ffac469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:10.641628 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0aee01-ee8c-4b3c-8b5a-c3db9ffac469 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:10.641915 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0aee01-ee8c-4b3c-8b5a-c3db9ffac469 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:10.642196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0aee01-ee8c-4b3c-8b5a-c3db9ffac469 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:10.643009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0aee01-ee8c-4b3c-8b5a-c3db9ffac469 None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:10.647364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0aee01-ee8c-4b3c-8b5a-c3db9ffac469 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:10.648892 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0aee01-ee8c-4b3c-8b5a-c3db9ffac469 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:10.663610 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13187c-6bef-4b62-9b28-bfabc263da37 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:10.663915 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13187c-6bef-4b62-9b28-bfabc263da37 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:10.664185 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13187c-6bef-4b62-9b28-bfabc263da37 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:10.665338 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 206/427] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:25:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:10.665776 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 222/428] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 31 14:25:10.675695 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b13187c-6bef-4b62-9b28-bfabc263da37 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:10.675888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af281cbe-228d-4396-b4ef-ab0c61289b41 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:10.676186 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af281cbe-228d-4396-b4ef-ab0c61289b41 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:10.676468 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af281cbe-228d-4396-b4ef-ab0c61289b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:10.715607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-af281cbe-228d-4396-b4ef-ab0c61289b41 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:10.721236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-af281cbe-228d-4396-b4ef-ab0c61289b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:10.921613 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b13187c-6bef-4b62-9b28-bfabc263da37 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:10.926044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 207/429] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:10.960909 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af281cbe-228d-4396-b4ef-ab0c61289b41 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:10.986220 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9df0e-4b17-4d2c-bdd7-92b11e09a4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:10.986909 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9df0e-4b17-4d2c-bdd7-92b11e09a4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:10.987191 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9df0e-4b17-4d2c-bdd7-92b11e09a4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:10.987461 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9df0e-4b17-4d2c-bdd7-92b11e09a4d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:11.018484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:11.042491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:11.093910 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d9df0e-4b17-4d2c-bdd7-92b11e09a4d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:11.095435 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d9df0e-4b17-4d2c-bdd7-92b11e09a4d2 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:11.163081 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af281cbe-228d-4396-b4ef-ab0c61289b41 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:11.204158 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 223/430] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:25:11.209728 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9df0e-4b17-4d2c-bdd7-92b11e09a4d2 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:11.248278 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b72679-cfde-411b-b25f-cb105634a928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:11.249458 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b72679-cfde-411b-b25f-cb105634a928 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:11.249937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b72679-cfde-411b-b25f-cb105634a928 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:11.250401 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b72679-cfde-411b-b25f-cb105634a928 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:11.272238 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b72679-cfde-411b-b25f-cb105634a928 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:11.281979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e5b72679-cfde-411b-b25f-cb105634a928 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 31 14:25:11.281979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:11.281979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:11.281979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:11.281979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:11.281979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:11.281979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:11.281979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:11.281979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:11.281979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:11.281979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:11.281979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:11.281979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:11.281979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:11.281979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:11.281979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:11.282888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:11.282888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:11.282888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:11.282888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:11.282888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:11.282888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:11.282888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:11.282888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:11.282888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:11.282888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:11.282888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:11.282888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:11.282888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:11.282888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:11.282888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:11.282888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:11.283757 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:11.283757 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:11.283757 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:11.283757 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:11.283757 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:11.283757 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:11.283757 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:11.283757 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:11.283757 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:11.283757 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:11.283757 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 31 14:25:11.283757 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:11.291329 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 208/431] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:25:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:11.292905 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b72679-cfde-411b-b25f-cb105634a928 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:11.306726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:11.351878 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9df0e-4b17-4d2c-bdd7-92b11e09a4d2 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:11.384947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:11.391154 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dccce63-8c90-47aa-9d90-a3421286d595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:11.392140 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccce63-8c90-47aa-9d90-a3421286d595 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:11.392713 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccce63-8c90-47aa-9d90-a3421286d595 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:11.393089 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccce63-8c90-47aa-9d90-a3421286d595 None admin] PATH_INFO: `/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:11.394667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dccce63-8c90-47aa-9d90-a3421286d595 None admin] RBAC: Authorizing `identity:get_project(project_id=e3f26f512b354f6ebdbf6c8d3cc3e472)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:11.396347 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dccce63-8c90-47aa-9d90-a3421286d595 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:11.402868 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 224/432] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:25:11 2022] GET /identity/v3/projects/e3f26f512b354f6ebdbf6c8d3cc3e472 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:11.403329 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b72679-cfde-411b-b25f-cb105634a928 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:11.403963 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dccce63-8c90-47aa-9d90-a3421286d595 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:11.439970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c656ed-4cb0-4bb6-8f47-b5e26a952c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:11.440969 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c656ed-4cb0-4bb6-8f47-b5e26a952c27 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:11.441500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c656ed-4cb0-4bb6-8f47-b5e26a952c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:11.441949 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c656ed-4cb0-4bb6-8f47-b5e26a952c27 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:11.442887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c656ed-4cb0-4bb6-8f47-b5e26a952c27 None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:11.443390 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c656ed-4cb0-4bb6-8f47-b5e26a952c27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:11.444918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c656ed-4cb0-4bb6-8f47-b5e26a952c27 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:11.459461 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 209/433] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:25:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:11.460887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c656ed-4cb0-4bb6-8f47-b5e26a952c27 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:11.514449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dccce63-8c90-47aa-9d90-a3421286d595 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:11.542461 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:11.552751 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6998f-6138-466e-91bd-7fed2dc30cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:11.553747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6998f-6138-466e-91bd-7fed2dc30cad None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:11.554140 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6998f-6138-466e-91bd-7fed2dc30cad None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:11.554488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6998f-6138-466e-91bd-7fed2dc30cad None admin] PATH_INFO: `/v3/groups/93ca2d3407df4b40a423023aec0efbbe` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:11.555938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6998f-6138-466e-91bd-7fed2dc30cad None admin] RBAC: Authorizing `identity:get_group(group_id=93ca2d3407df4b40a423023aec0efbbe)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:11.557606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6998f-6138-466e-91bd-7fed2dc30cad None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:11.564949 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 225/434] 10.222.0.67 () {56 vars in 1180 bytes} [Tue May 31 14:25:11 2022] GET /identity/v3/groups/93ca2d3407df4b40a423023aec0efbbe => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:11.565990 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6998f-6138-466e-91bd-7fed2dc30cad None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:11.573891 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c656ed-4cb0-4bb6-8f47-b5e26a952c27 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:11.608902 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8e005-0d97-4dd3-93e2-478201ca50f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:11.609927 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8e005-0d97-4dd3-93e2-478201ca50f0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:11.610407 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8e005-0d97-4dd3-93e2-478201ca50f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:11.610916 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8e005-0d97-4dd3-93e2-478201ca50f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:11.629903 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:11.676383 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6998f-6138-466e-91bd-7fed2dc30cad None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:11.682198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f8e005-0d97-4dd3-93e2-478201ca50f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:11.684459 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f8e005-0d97-4dd3-93e2-478201ca50f0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:11.712983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526aabf0-8f56-48f1-a53e-9d6574fc2c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:11.713566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526aabf0-8f56-48f1-a53e-9d6574fc2c30 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:11.713867 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526aabf0-8f56-48f1-a53e-9d6574fc2c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:11.714173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526aabf0-8f56-48f1-a53e-9d6574fc2c30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:11.715066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526aabf0-8f56-48f1-a53e-9d6574fc2c30 None admin] RBAC: Adding query filter params (group.id=93ca2d3407df4b40a423023aec0efbbe, role.id=9ce8fdecf52d482d894fdeec4f530624, scope.project.id=e3f26f512b354f6ebdbf6c8d3cc3e472) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:11.715384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526aabf0-8f56-48f1-a53e-9d6574fc2c30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:11.717038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526aabf0-8f56-48f1-a53e-9d6574fc2c30 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:11.731657 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 226/435] 10.222.0.67 () {56 vars in 1376 bytes} [Tue May 31 14:25:11 2022] GET /identity/v3/role_assignments?group.id=93ca2d3407df4b40a423023aec0efbbe&scope.project.id=e3f26f512b354f6ebdbf6c8d3cc3e472&role.id=9ce8fdecf52d482d894fdeec4f530624 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:11.804429 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 210/436] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 31 14:25:11.806186 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526aabf0-8f56-48f1-a53e-9d6574fc2c30 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:11.869331 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:11.916494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526aabf0-8f56-48f1-a53e-9d6574fc2c30 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:11.955949 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de50e25-948f-4754-88a8-9c5c6d8fcecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:11.956808 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de50e25-948f-4754-88a8-9c5c6d8fcecb None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:11.957087 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de50e25-948f-4754-88a8-9c5c6d8fcecb None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:11.957364 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de50e25-948f-4754-88a8-9c5c6d8fcecb None admin] PATH_INFO: `/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:11.958538 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de50e25-948f-4754-88a8-9c5c6d8fcecb None admin] RBAC: Authorizing `identity:get_project(project_id=bef8d3a846e34a8daf7eecb46a5fd2f8)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:11.960060 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de50e25-948f-4754-88a8-9c5c6d8fcecb None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:11.961904 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 227/437] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:25:11 2022] GET /identity/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:11.967939 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8e005-0d97-4dd3-93e2-478201ca50f0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:12.022176 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:12.084171 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8e005-0d97-4dd3-93e2-478201ca50f0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:12.109648 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee210a-0841-46c7-9666-5cc32f9aa6ee None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:12.109967 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee210a-0841-46c7-9666-5cc32f9aa6ee None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:12.110213 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee210a-0841-46c7-9666-5cc32f9aa6ee None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:12.111884 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 228/438] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:12.116978 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7a1a4-77b5-4aae-a8e2-85d9461fcb48 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:12.117273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7a1a4-77b5-4aae-a8e2-85d9461fcb48 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:12.117558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7a1a4-77b5-4aae-a8e2-85d9461fcb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:12.129936 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfe4b0-64d3-4f82-938f-5038251a47e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:12.130926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfe4b0-64d3-4f82-938f-5038251a47e2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:12.131213 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfe4b0-64d3-4f82-938f-5038251a47e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:12.131495 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfe4b0-64d3-4f82-938f-5038251a47e2 None admin] PATH_INFO: `/v3/groups/93ca2d3407df4b40a423023aec0efbbe` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:12.133740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cfe4b0-64d3-4f82-938f-5038251a47e2 None admin] RBAC: Authorizing `identity:get_group(group_id=93ca2d3407df4b40a423023aec0efbbe)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:12.135832 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cfe4b0-64d3-4f82-938f-5038251a47e2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:12.137860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 211/439] 10.222.0.67 () {56 vars in 1180 bytes} [Tue May 31 14:25:11 2022] GET /identity/v3/groups/93ca2d3407df4b40a423023aec0efbbe => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:12.143727 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfe4b0-64d3-4f82-938f-5038251a47e2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:12.147713 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-9ca7a1a4-77b5-4aae-a8e2-85d9461fcb48 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:12.148957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9ca7a1a4-77b5-4aae-a8e2-85d9461fcb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:12.236001 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 229/440] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 31 14:25:12.260433 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7a1a4-77b5-4aae-a8e2-85d9461fcb48 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:12.281182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfe4b0-64d3-4f82-938f-5038251a47e2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:12.300853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:12.327682 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08c994-5c12-4bd0-8d98-5bb9a5c58155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:12.328876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08c994-5c12-4bd0-8d98-5bb9a5c58155 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:12.329318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08c994-5c12-4bd0-8d98-5bb9a5c58155 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:12.329738 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08c994-5c12-4bd0-8d98-5bb9a5c58155 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:12.330605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08c994-5c12-4bd0-8d98-5bb9a5c58155 None admin] RBAC: Adding query filter params (group.id=93ca2d3407df4b40a423023aec0efbbe, role.id=9ce8fdecf52d482d894fdeec4f530624, scope.project.id=bef8d3a846e34a8daf7eecb46a5fd2f8) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:12.331149 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08c994-5c12-4bd0-8d98-5bb9a5c58155 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:12.333110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08c994-5c12-4bd0-8d98-5bb9a5c58155 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:12.339869 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:12.343701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 212/441] 10.222.0.67 () {56 vars in 1376 bytes} [Tue May 31 14:25:12 2022] GET /identity/v3/role_assignments?group.id=93ca2d3407df4b40a423023aec0efbbe&scope.project.id=bef8d3a846e34a8daf7eecb46a5fd2f8&role.id=9ce8fdecf52d482d894fdeec4f530624 => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:12.406530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7a1a4-77b5-4aae-a8e2-85d9461fcb48 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:12.448688 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24dc601-9717-44f7-bef4-2e342e09efa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:12.449789 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24dc601-9717-44f7-bef4-2e342e09efa5 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:12.450212 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24dc601-9717-44f7-bef4-2e342e09efa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:12.450625 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24dc601-9717-44f7-bef4-2e342e09efa5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:12.463479 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24dc601-9717-44f7-bef4-2e342e09efa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:12.471987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-e24dc601-9717-44f7-bef4-2e342e09efa5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:25:12.471987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:12.471987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:12.471987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:12.471987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:12.471987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:12.471987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:12.471987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:12.471987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:12.471987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:12.471987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:12.471987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:12.471987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:12.471987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:12.471987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:12.471987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:12.473114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:12.473114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:12.473114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:12.473114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:12.473114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:12.473114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:12.473114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:12.473114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:12.473114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:12.473114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:12.473114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:12.473114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:12.473114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:12.473114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:12.473114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:12.473114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:12.474091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:12.474091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:12.474091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:12.474091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:12.474091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:12.474091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:12.474091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:12.474091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:12.474091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:12.474091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:12.474091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:25:12.474091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:12.477771 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 230/442] 10.222.0.67 () {56 vars in 1114 bytes} [Tue May 31 14:25:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:25:12.484990 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08c994-5c12-4bd0-8d98-5bb9a5c58155 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:12.596204 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08c994-5c12-4bd0-8d98-5bb9a5c58155 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:12.631844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e92d94-ae95-4865-ad04-6a0fe72dafcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:12.632524 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e92d94-ae95-4865-ad04-6a0fe72dafcd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:12.632845 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e92d94-ae95-4865-ad04-6a0fe72dafcd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:12.633132 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e92d94-ae95-4865-ad04-6a0fe72dafcd None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:12.633890 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e92d94-ae95-4865-ad04-6a0fe72dafcd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:12.634209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e92d94-ae95-4865-ad04-6a0fe72dafcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:12.635644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e92d94-ae95-4865-ad04-6a0fe72dafcd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:12.644302 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 213/443] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:12.649631 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24dc601-9717-44f7-bef4-2e342e09efa5 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:12.674724 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c24f3-d030-41cc-af37-da4d62678dec None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:12.675017 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c24f3-d030-41cc-af37-da4d62678dec None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:12.675298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c24f3-d030-41cc-af37-da4d62678dec None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:12.676670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 214/444] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:12.681671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e838e-d249-49c0-a2cc-51e23eeae0b1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:12.681958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e838e-d249-49c0-a2cc-51e23eeae0b1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:12.682241 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e838e-d249-49c0-a2cc-51e23eeae0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:12.701073 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0e6e838e-d249-49c0-a2cc-51e23eeae0b1 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:12.702532 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0e6e838e-d249-49c0-a2cc-51e23eeae0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:12.743014 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:12.765129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24dc601-9717-44f7-bef4-2e342e09efa5 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:12.778551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 215/445] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 31 14:25:12.785653 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:12.802467 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e838e-d249-49c0-a2cc-51e23eeae0b1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:12.805114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc4b38-5a07-4c81-a412-6d1b1c59c0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:12.806561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc4b38-5a07-4c81-a412-6d1b1c59c0aa None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:12.806868 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc4b38-5a07-4c81-a412-6d1b1c59c0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:12.807144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc4b38-5a07-4c81-a412-6d1b1c59c0aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:12.874703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc4b38-5a07-4c81-a412-6d1b1c59c0aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:12.876233 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc4b38-5a07-4c81-a412-6d1b1c59c0aa None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:12.907503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e838e-d249-49c0-a2cc-51e23eeae0b1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:12.941018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595f5b2-49c3-467d-b55f-c67cad6134dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:12.941816 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595f5b2-49c3-467d-b55f-c67cad6134dd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:12.942107 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595f5b2-49c3-467d-b55f-c67cad6134dd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:12.942388 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595f5b2-49c3-467d-b55f-c67cad6134dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:12.952683 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f595f5b2-49c3-467d-b55f-c67cad6134dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:12.960330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f595f5b2-49c3-467d-b55f-c67cad6134dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:25:12.960330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:12.960330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:12.960330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:12.960330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:12.960330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:12.960330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:12.960330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:12.960330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:12.960330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:12.960330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:12.960330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:12.960330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:12.960330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:12.960330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:12.960330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:12.961269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:12.961269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:12.961269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:12.961269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:12.961269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:12.961269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:12.961269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:12.961269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:12.961269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:12.961269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:12.961269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:12.961269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:12.961269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:12.961269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:12.961269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:12.961269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:12.962060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:12.962060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:12.962060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:12.962060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:12.962060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:12.962060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:12.962060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:12.962060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:12.962060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:12.962060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:12.962060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:25:12.962060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:12.962060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 216/446] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:25:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:25:12.967682 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595f5b2-49c3-467d-b55f-c67cad6134dd None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:12.974152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 231/447] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:25:12.979626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc4b38-5a07-4c81-a412-6d1b1c59c0aa None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:12.996331 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:12.998755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:13.066814 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595f5b2-49c3-467d-b55f-c67cad6134dd None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:13.077885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc4b38-5a07-4c81-a412-6d1b1c59c0aa None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:13.100090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339500b-a77c-435c-81fb-c20c8839e851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:13.100750 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339500b-a77c-435c-81fb-c20c8839e851 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:13.101041 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339500b-a77c-435c-81fb-c20c8839e851 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:13.101316 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339500b-a77c-435c-81fb-c20c8839e851 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:13.102060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339500b-a77c-435c-81fb-c20c8839e851 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:13.102374 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339500b-a77c-435c-81fb-c20c8839e851 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:13.103784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339500b-a77c-435c-81fb-c20c8839e851 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:13.111438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d07766-cc0a-4224-8843-ca1a6dafcbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:13.111606 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 217/448] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:25:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:25:13.112365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d07766-cc0a-4224-8843-ca1a6dafcbe7 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:13.112644 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d07766-cc0a-4224-8843-ca1a6dafcbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:13.112998 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d07766-cc0a-4224-8843-ca1a6dafcbe7 None admin] PATH_INFO: `/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:13.114165 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d07766-cc0a-4224-8843-ca1a6dafcbe7 None admin] RBAC: Authorizing `identity:get_project(project_id=bef8d3a846e34a8daf7eecb46a5fd2f8)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:13.115617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d07766-cc0a-4224-8843-ca1a6dafcbe7 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:13.117124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339500b-a77c-435c-81fb-c20c8839e851 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:13.121602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 232/449] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:25:12 2022] GET /identity/v3/projects/bef8d3a846e34a8daf7eecb46a5fd2f8 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:25:13.122767 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d07766-cc0a-4224-8843-ca1a6dafcbe7 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:13.203863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:13.215885 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:13.223054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d07766-cc0a-4224-8843-ca1a6dafcbe7 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:13.224941 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339500b-a77c-435c-81fb-c20c8839e851 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:13.257111 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890a403-6970-4a54-9a94-c9c6c708b1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:13.257756 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890a403-6970-4a54-9a94-c9c6c708b1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:13.258038 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890a403-6970-4a54-9a94-c9c6c708b1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:13.258309 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890a403-6970-4a54-9a94-c9c6c708b1d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:13.268272 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec386dd-15a3-49dc-a01d-db0897dadcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:13.269552 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec386dd-15a3-49dc-a01d-db0897dadcff None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:13.269977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec386dd-15a3-49dc-a01d-db0897dadcff None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:13.270396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec386dd-15a3-49dc-a01d-db0897dadcff None admin] PATH_INFO: `/v3/groups/93ca2d3407df4b40a423023aec0efbbe` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:13.272156 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec386dd-15a3-49dc-a01d-db0897dadcff None admin] RBAC: Authorizing `identity:get_group(group_id=93ca2d3407df4b40a423023aec0efbbe)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:13.274511 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec386dd-15a3-49dc-a01d-db0897dadcff None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:13.276484 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 233/450] 10.222.0.67 () {56 vars in 1180 bytes} [Tue May 31 14:25:13 2022] GET /identity/v3/groups/93ca2d3407df4b40a423023aec0efbbe => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:25:13.282716 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec386dd-15a3-49dc-a01d-db0897dadcff None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:13.325288 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f890a403-6970-4a54-9a94-c9c6c708b1d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:13.327025 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f890a403-6970-4a54-9a94-c9c6c708b1d4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:13.378409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec386dd-15a3-49dc-a01d-db0897dadcff None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:13.411332 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bfa865-2393-4264-8afb-7830b78969b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:13.411915 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bfa865-2393-4264-8afb-7830b78969b6 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:13.412174 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bfa865-2393-4264-8afb-7830b78969b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:13.412468 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bfa865-2393-4264-8afb-7830b78969b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:13.413286 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bfa865-2393-4264-8afb-7830b78969b6 None admin] RBAC: Adding query filter params (group.id=93ca2d3407df4b40a423023aec0efbbe, role.id=1b9c18d4b8b540bf9825080748cec3bc, scope.project.id=bef8d3a846e34a8daf7eecb46a5fd2f8) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:13.413601 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bfa865-2393-4264-8afb-7830b78969b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:13.415050 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bfa865-2393-4264-8afb-7830b78969b6 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:13.415785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:13.421503 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 234/451] 10.222.0.67 () {56 vars in 1376 bytes} [Tue May 31 14:25:13 2022] GET /identity/v3/role_assignments?group.id=93ca2d3407df4b40a423023aec0efbbe&scope.project.id=bef8d3a846e34a8daf7eecb46a5fd2f8&role.id=1b9c18d4b8b540bf9825080748cec3bc => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:25:13.426697 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 218/452] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 31 14:25:13.429131 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bfa865-2393-4264-8afb-7830b78969b6 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:13.448804 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:13.535453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bfa865-2393-4264-8afb-7830b78969b6 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:13.572946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb921e-d55e-4e53-ba07-d7553d2f365e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:13.575010 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb921e-d55e-4e53-ba07-d7553d2f365e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:13.575503 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb921e-d55e-4e53-ba07-d7553d2f365e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:13.575973 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb921e-d55e-4e53-ba07-d7553d2f365e None admin] PATH_INFO: `/v3/projects/929ceca16868457ca23cb37daf0ee36d` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:13.578163 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb921e-d55e-4e53-ba07-d7553d2f365e None admin] RBAC: Authorizing `identity:get_project(project_id=929ceca16868457ca23cb37daf0ee36d)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:13.579738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb921e-d55e-4e53-ba07-d7553d2f365e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:13.581887 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 235/453] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:25:13 2022] GET /identity/v3/projects/929ceca16868457ca23cb37daf0ee36d => generated 344 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:13.587431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890a403-6970-4a54-9a94-c9c6c708b1d4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:13.690140 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890a403-6970-4a54-9a94-c9c6c708b1d4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:13.721603 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cda11c-af55-4dd0-a6a6-600a732ab503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:13.722457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cda11c-af55-4dd0-a6a6-600a732ab503 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:13.722767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cda11c-af55-4dd0-a6a6-600a732ab503 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:13.723044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cda11c-af55-4dd0-a6a6-600a732ab503 None admin] PATH_INFO: `/v3/groups/dcc5a08271114a7abb972516ac9472fc` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:13.724205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cda11c-af55-4dd0-a6a6-600a732ab503 None admin] RBAC: Authorizing `identity:get_group(group_id=dcc5a08271114a7abb972516ac9472fc)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:13.725627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cda11c-af55-4dd0-a6a6-600a732ab503 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:13.727608 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 219/454] 10.222.0.67 () {56 vars in 1180 bytes} [Tue May 31 14:25:13 2022] GET /identity/v3/groups/dcc5a08271114a7abb972516ac9472fc => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:25:13.733521 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb921e-d55e-4e53-ba07-d7553d2f365e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:13.820041 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:13.836183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb921e-d55e-4e53-ba07-d7553d2f365e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:13.868593 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd3c44-8f7e-4b38-9817-d725eb75945d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:13.869236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd3c44-8f7e-4b38-9817-d725eb75945d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:13.869518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd3c44-8f7e-4b38-9817-d725eb75945d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:13.869793 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd3c44-8f7e-4b38-9817-d725eb75945d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:13.870618 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd3c44-8f7e-4b38-9817-d725eb75945d None admin] RBAC: Adding query filter params (group.id=dcc5a08271114a7abb972516ac9472fc, role.id=e8d679596ffb45939229d1e2a13ce9b0, scope.project.id=929ceca16868457ca23cb37daf0ee36d) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:13.870962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd3c44-8f7e-4b38-9817-d725eb75945d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:13.873031 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd3c44-8f7e-4b38-9817-d725eb75945d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:13.878612 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 236/455] 10.222.0.67 () {56 vars in 1376 bytes} [Tue May 31 14:25:13 2022] GET /identity/v3/role_assignments?group.id=dcc5a08271114a7abb972516ac9472fc&scope.project.id=929ceca16868457ca23cb37daf0ee36d&role.id=e8d679596ffb45939229d1e2a13ce9b0 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:16.661957 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197970a-f15c-4eb5-ba76-27e5b76c5f7b None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:16.663036 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197970a-f15c-4eb5-ba76-27e5b76c5f7b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:16.663534 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197970a-f15c-4eb5-ba76-27e5b76c5f7b None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:16.665612 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 220/456] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:16.671328 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920965cd-71a3-4ddd-aaf2-d26fd1d17526 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:16.672461 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920965cd-71a3-4ddd-aaf2-d26fd1d17526 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:16.672947 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920965cd-71a3-4ddd-aaf2-d26fd1d17526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:16.693038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-920965cd-71a3-4ddd-aaf2-d26fd1d17526 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:16.694586 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-920965cd-71a3-4ddd-aaf2-d26fd1d17526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:16.744052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bf75d-e137-4198-81c3-a6887a0c8066 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:16.744052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bf75d-e137-4198-81c3-a6887a0c8066 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:16.744052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bf75d-e137-4198-81c3-a6887a0c8066 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:16.744652 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 221/457] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:16.751030 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d73a83-eba3-4cb2-bafa-60942ccff0ed None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:16.751584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d73a83-eba3-4cb2-bafa-60942ccff0ed None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:16.752089 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d73a83-eba3-4cb2-bafa-60942ccff0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:16.784472 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-03d73a83-eba3-4cb2-bafa-60942ccff0ed None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:16.785693 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-03d73a83-eba3-4cb2-bafa-60942ccff0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:16.812135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 237/458] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:16.839777 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920965cd-71a3-4ddd-aaf2-d26fd1d17526 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:16.904428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:16.927992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:16.933014 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 222/459] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:16.952957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920965cd-71a3-4ddd-aaf2-d26fd1d17526 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:16.970486 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d73a83-eba3-4cb2-bafa-60942ccff0ed None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:16.996844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383a9b6-182b-4e84-b0f2-b7b4f77eea4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:16.997766 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383a9b6-182b-4e84-b0f2-b7b4f77eea4e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:16.998055 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383a9b6-182b-4e84-b0f2-b7b4f77eea4e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:16.998330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383a9b6-182b-4e84-b0f2-b7b4f77eea4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:17.010068 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383a9b6-182b-4e84-b0f2-b7b4f77eea4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:17.011723 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383a9b6-182b-4e84-b0f2-b7b4f77eea4e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:17.030749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-f383a9b6-182b-4e84-b0f2-b7b4f77eea4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:17.030749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:17.030749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:17.030749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:17.030749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:17.030749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:17.030749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:17.030749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:17.030749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:17.030749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:17.030749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:17.030749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:17.030749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:17.030749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:17.030749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:17.031646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:17.031646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:17.031646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:17.031646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:17.031646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:17.031646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:17.031646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:17.031646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:17.031646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:17.031646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:17.031646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:17.031646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:17.031646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:17.031646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:17.031646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:17.031646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:17.031646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:17.032515 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:17.032515 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:17.032515 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:17.032515 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:17.032515 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 238/460] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:25:17.033482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8b1d1-1f14-482a-9e89-bff22d656ab5 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:17.036658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8b1d1-1f14-482a-9e89-bff22d656ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:17.036658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8b1d1-1f14-482a-9e89-bff22d656ab5 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:17.036658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 239/461] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:17.040457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8b1d1-1f14-482a-9e89-bff22d656ab5 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:17.110125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d73a83-eba3-4cb2-bafa-60942ccff0ed None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:17.155318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48206010-b3c1-487c-89c7-0b1ff8640a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:17.156400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48206010-b3c1-487c-89c7-0b1ff8640a4b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:17.156877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48206010-b3c1-487c-89c7-0b1ff8640a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:17.157339 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48206010-b3c1-487c-89c7-0b1ff8640a4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:17.160033 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8b1d1-1f14-482a-9e89-bff22d656ab5 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:17.169072 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48206010-b3c1-487c-89c7-0b1ff8640a4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:17.170842 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48206010-b3c1-487c-89c7-0b1ff8640a4b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:17.189823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-48206010-b3c1-487c-89c7-0b1ff8640a4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:17.189823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:17.189823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:17.189823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:17.189823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:17.189823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:17.189823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:17.189823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:17.189823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:17.189823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:17.189823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:17.189823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:17.189823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:17.189823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:17.189823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:17.190690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:17.190690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:17.190690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:17.190690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:17.190690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:17.190690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:17.190690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:17.190690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:17.190690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:17.190690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:17.190690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:17.190690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:17.190690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:17.190690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:17.190690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:17.190690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:17.190690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:17.191542 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:17.191542 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:17.191542 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:17.191542 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:17.193646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:17.193823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 223/462] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:25:17.195431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbb9b1-dcbe-4ec0-9198-b88757445546 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:17.195928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbb9b1-dcbe-4ec0-9198-b88757445546 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:17.196392 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbb9b1-dcbe-4ec0-9198-b88757445546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:17.201353 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb80ba4-bb35-433f-a2a4-7a13b5f7c67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:17.202285 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb80ba4-bb35-433f-a2a4-7a13b5f7c67b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:17.202574 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb80ba4-bb35-433f-a2a4-7a13b5f7c67b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:17.202887 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb80ba4-bb35-433f-a2a4-7a13b5f7c67b None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:17.203811 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb80ba4-bb35-433f-a2a4-7a13b5f7c67b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:17.204126 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb80ba4-bb35-433f-a2a4-7a13b5f7c67b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:17.206382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb80ba4-bb35-433f-a2a4-7a13b5f7c67b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:17.216355 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3acbb9b1-dcbe-4ec0-9198-b88757445546 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:17.217778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3acbb9b1-dcbe-4ec0-9198-b88757445546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:17.228609 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 240/463] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:17.229462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb80ba4-bb35-433f-a2a4-7a13b5f7c67b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:17.260593 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:17.290182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 224/464] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:17.291406 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbb9b1-dcbe-4ec0-9198-b88757445546 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:17.352797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb80ba4-bb35-433f-a2a4-7a13b5f7c67b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:17.389459 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba7d15-379c-40e6-9b30-71ab525b8cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:17.390517 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba7d15-379c-40e6-9b30-71ab525b8cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:17.391059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba7d15-379c-40e6-9b30-71ab525b8cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:17.391525 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba7d15-379c-40e6-9b30-71ab525b8cd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:17.392433 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba7d15-379c-40e6-9b30-71ab525b8cd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:17.392948 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba7d15-379c-40e6-9b30-71ab525b8cd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:17.394549 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba7d15-379c-40e6-9b30-71ab525b8cd3 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:17.407772 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 241/465] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:17.408990 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba7d15-379c-40e6-9b30-71ab525b8cd3 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:17.418026 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbb9b1-dcbe-4ec0-9198-b88757445546 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:17.463414 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:17.466823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d838bd6-7eb5-4ec5-98c5-7ff39e81f340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:17.467668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d838bd6-7eb5-4ec5-98c5-7ff39e81f340 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:17.468115 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d838bd6-7eb5-4ec5-98c5-7ff39e81f340 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:17.468523 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d838bd6-7eb5-4ec5-98c5-7ff39e81f340 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:17.469383 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d838bd6-7eb5-4ec5-98c5-7ff39e81f340 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:17.470966 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d838bd6-7eb5-4ec5-98c5-7ff39e81f340 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:17.502959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 225/466] 10.222.0.67 () {60 vars in 1132 bytes} [Tue May 31 14:25:17 2022] POST /identity/v3/users => generated 246 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:25:17.502959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d838bd6-7eb5-4ec5-98c5-7ff39e81f340 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:17.533725 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba7d15-379c-40e6-9b30-71ab525b8cd3 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:17.546185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:17.588664 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6a272-f337-456f-a5c6-6a8dc8adc42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:17.590877 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6a272-f337-456f-a5c6-6a8dc8adc42c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:17.591438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6a272-f337-456f-a5c6-6a8dc8adc42c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:17.592163 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6a272-f337-456f-a5c6-6a8dc8adc42c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:17.608933 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d6a272-f337-456f-a5c6-6a8dc8adc42c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:17.610789 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d6a272-f337-456f-a5c6-6a8dc8adc42c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:17.630945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-d2d6a272-f337-456f-a5c6-6a8dc8adc42c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:17.630945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:17.630945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:17.630945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:17.630945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:17.630945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:17.630945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:17.630945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:17.630945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:17.630945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:17.630945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:17.630945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:17.630945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:17.630945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:17.630945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:17.631866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:17.631866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:17.631866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:17.631866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:17.631866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:17.631866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:17.631866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:17.631866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:17.631866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:17.631866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:17.631866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:17.631866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:17.631866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:17.631866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:17.631866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:17.631866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:17.631866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:17.632702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:17.632702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:17.632702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:17.632702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:17.634517 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 242/467] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:17.636351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac74b0-bb53-48ca-a766-6266f3e85469 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:17.637376 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac74b0-bb53-48ca-a766-6266f3e85469 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:17.637858 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac74b0-bb53-48ca-a766-6266f3e85469 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:17.644317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d838bd6-7eb5-4ec5-98c5-7ff39e81f340 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:17.644502 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 243/468] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:17.645363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac74b0-bb53-48ca-a766-6266f3e85469 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:17.690316 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0759e-7344-467f-b4d1-2cd71afd975e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:17.691680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0759e-7344-467f-b4d1-2cd71afd975e None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:17.692144 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0759e-7344-467f-b4d1-2cd71afd975e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:17.692570 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0759e-7344-467f-b4d1-2cd71afd975e None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:17.693693 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0759e-7344-467f-b4d1-2cd71afd975e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:17.695659 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0759e-7344-467f-b4d1-2cd71afd975e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:17.723480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 226/469] 10.222.0.67 () {60 vars in 1132 bytes} [Tue May 31 14:25:17 2022] POST /identity/v3/users => generated 248 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:25:17.724975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29ccd9-c8b6-4ba8-ae26-2da9f71bdb6f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:17.725405 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29ccd9-c8b6-4ba8-ae26-2da9f71bdb6f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:17.725818 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29ccd9-c8b6-4ba8-ae26-2da9f71bdb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:17.744946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-df29ccd9-c8b6-4ba8-ae26-2da9f71bdb6f None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:17.746450 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-df29ccd9-c8b6-4ba8-ae26-2da9f71bdb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:17.797887 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac74b0-bb53-48ca-a766-6266f3e85469 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:17.806287 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:17.848590 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 227/470] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:17.853865 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a74ad6-f614-49b4-bbbf-a731e137a8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:17.853865 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a74ad6-f614-49b4-bbbf-a731e137a8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:17.853865 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a74ad6-f614-49b4-bbbf-a731e137a8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:17.853865 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a74ad6-f614-49b4-bbbf-a731e137a8e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:17.854232 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a74ad6-f614-49b4-bbbf-a731e137a8e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:17.854232 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a74ad6-f614-49b4-bbbf-a731e137a8e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:17.854232 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a74ad6-f614-49b4-bbbf-a731e137a8e1 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:17.876588 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29ccd9-c8b6-4ba8-ae26-2da9f71bdb6f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:17.884070 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 244/471] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:17.893994 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a74ad6-f614-49b4-bbbf-a731e137a8e1 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:17.895991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:18.009431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29ccd9-c8b6-4ba8-ae26-2da9f71bdb6f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:18.021279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a74ad6-f614-49b4-bbbf-a731e137a8e1 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:18.059411 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744cce1f-ed07-4681-8b20-705f40cc9b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:18.061111 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744cce1f-ed07-4681-8b20-705f40cc9b90 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:18.061850 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744cce1f-ed07-4681-8b20-705f40cc9b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:18.062307 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744cce1f-ed07-4681-8b20-705f40cc9b90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:18.071363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67544bba-1281-4df5-9f57-655e9b6cd947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:18.072240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67544bba-1281-4df5-9f57-655e9b6cd947 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:18.072524 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67544bba-1281-4df5-9f57-655e9b6cd947 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:18.072799 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67544bba-1281-4df5-9f57-655e9b6cd947 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:18.073521 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67544bba-1281-4df5-9f57-655e9b6cd947 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:18.075197 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67544bba-1281-4df5-9f57-655e9b6cd947 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:18.077036 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744cce1f-ed07-4681-8b20-705f40cc9b90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:18.078752 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744cce1f-ed07-4681-8b20-705f40cc9b90 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:18.098730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-744cce1f-ed07-4681-8b20-705f40cc9b90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:18.098730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:18.098730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:18.098730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:18.098730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:18.098730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:18.098730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:18.098730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:18.098730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:18.098730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:18.098730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:18.098730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:18.098730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:18.098730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:18.098730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:18.099672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:18.099672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:18.099672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:18.099672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:18.099672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:18.099672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:18.099672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:18.099672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:18.099672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:18.099672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:18.099672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:18.099672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:18.099672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:18.099672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:18.099672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:18.099672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:18.099672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:18.100512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:18.100512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:18.100512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:18.100512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:18.100733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 245/472] 10.222.0.67 () {60 vars in 1132 bytes} [Tue May 31 14:25:17 2022] POST /identity/v3/users => generated 248 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:25:18.100812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 228/473] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:25:18.100879 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6136576-aac9-4297-a8e5-4d9e114f09a8 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:18.100879 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6136576-aac9-4297-a8e5-4d9e114f09a8 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:18.101010 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6136576-aac9-4297-a8e5-4d9e114f09a8 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:18.102517 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 246/474] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:18.108618 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e8256-b877-483b-a53f-679434697984 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:18.109009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e8256-b877-483b-a53f-679434697984 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:18.109362 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e8256-b877-483b-a53f-679434697984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:18.114503 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6136576-aac9-4297-a8e5-4d9e114f09a8 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:18.139391 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-635e8256-b877-483b-a53f-679434697984 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:18.140652 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-635e8256-b877-483b-a53f-679434697984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:18.157970 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:18.275971 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:18.283276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 229/475] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:18.321751 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6136576-aac9-4297-a8e5-4d9e114f09a8 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:18.340902 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e8256-b877-483b-a53f-679434697984 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:18.369477 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03080833-dd78-45c7-a69c-426ff13424bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:18.370319 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03080833-dd78-45c7-a69c-426ff13424bc None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:18.370607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03080833-dd78-45c7-a69c-426ff13424bc None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:18.370915 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03080833-dd78-45c7-a69c-426ff13424bc None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:18.372055 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03080833-dd78-45c7-a69c-426ff13424bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:18.372372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03080833-dd78-45c7-a69c-426ff13424bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:18.374240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03080833-dd78-45c7-a69c-426ff13424bc None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:18.390757 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 247/476] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:18.395565 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03080833-dd78-45c7-a69c-426ff13424bc None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:18.463171 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e8256-b877-483b-a53f-679434697984 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:18.499039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:18.500938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e5064-05f4-4459-ac3c-4810a1d82a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:18.501781 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e5064-05f4-4459-ac3c-4810a1d82a8c None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:18.502228 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e5064-05f4-4459-ac3c-4810a1d82a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:18.502737 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e5064-05f4-4459-ac3c-4810a1d82a8c None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:18.503660 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e5064-05f4-4459-ac3c-4810a1d82a8c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:18.505223 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e5064-05f4-4459-ac3c-4810a1d82a8c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:18.514162 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.sql.core [None req-9d6e5064-05f4-4459-ac3c-4810a1d82a8c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 31 14:25:18.514162 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 31 14:25:18.514162 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [parameters: {'id': 'caddcfaa26954cf793620da9181d88f6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 31 14:25:18.514162 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80233) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 31 14:25:18.517113 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03080833-dd78-45c7-a69c-426ff13424bc None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:18.536424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-9d6e5064-05f4-4459-ac3c-4810a1d82a8c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 31 14:25:18.536424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:18.536424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 31 14:25:18.536424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.dialect.do_execute( May 31 14:25:18.536424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 31 14:25:18.536424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 31 14:25:18.536424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 31 14:25:18.536424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result = self._query(query) May 31 14:25:18.536424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 31 14:25:18.536424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application conn.query(q) May 31 14:25:18.536424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 31 14:25:18.536424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 31 14:25:18.536424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 31 14:25:18.536424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result.read() May 31 14:25:18.536424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 31 14:25:18.537412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 31 14:25:18.537412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 31 14:25:18.537412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application packet.raise_for_error() May 31 14:25:18.537412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 31 14:25:18.537412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 31 14:25:18.537412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 31 14:25:18.537412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 31 14:25:18.537412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 31 14:25:18.537412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:18.537412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 31 14:25:18.537412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:18.537412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:18.537412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 31 14:25:18.537412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 31 14:25:18.537412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 31 14:25:18.537412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return ref.to_dict() May 31 14:25:18.537412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 31 14:25:18.537412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application next(self.gen) May 31 14:25:18.538345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 31 14:25:18.538345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application yield resource May 31 14:25:18.538345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 31 14:25:18.538345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application next(self.gen) May 31 14:25:18.538345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 31 14:25:18.538345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.session.rollback() May 31 14:25:18.538345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 31 14:25:18.538345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.force_reraise() May 31 14:25:18.538345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 31 14:25:18.538345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise self.value May 31 14:25:18.538345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 31 14:25:18.538345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 31 14:25:18.538345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 31 14:25:18.538345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application session.commit() May 31 14:25:18.538345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 31 14:25:18.538345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 31 14:25:18.538345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 31 14:25:18.539237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._prepare_impl() May 31 14:25:18.539237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 31 14:25:18.539237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.session.flush() May 31 14:25:18.539237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 31 14:25:18.539237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._flush(objects) May 31 14:25:18.539237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 31 14:25:18.539237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 31 14:25:18.539237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 31 14:25:18.539237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application compat.raise_( May 31 14:25:18.539237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 31 14:25:18.539237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception May 31 14:25:18.539237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 31 14:25:18.539237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application flush_context.execute() May 31 14:25:18.539237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 31 14:25:18.539237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rec.execute(self) May 31 14:25:18.539237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 31 14:25:18.539237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 31 14:25:18.540090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 31 14:25:18.540090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application _emit_insert_statements( May 31 14:25:18.540090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 31 14:25:18.540090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application c = connection._execute_20( May 31 14:25:18.540090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 31 14:25:18.540090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 31 14:25:18.540090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 31 14:25:18.540090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 31 14:25:18.540090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 31 14:25:18.540090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ret = self._execute_context( May 31 14:25:18.540090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 31 14:25:18.540090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 31 14:25:18.540090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 31 14:25:18.540090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 31 14:25:18.540090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 31 14:25:18.540090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception May 31 14:25:18.543209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 31 14:25:18.543209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.dialect.do_execute( May 31 14:25:18.543209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 31 14:25:18.543209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 31 14:25:18.543209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 31 14:25:18.543209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result = self._query(query) May 31 14:25:18.543209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 31 14:25:18.543209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application conn.query(q) May 31 14:25:18.543209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 31 14:25:18.543209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 31 14:25:18.543209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 31 14:25:18.543209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result.read() May 31 14:25:18.543209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 31 14:25:18.543209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 31 14:25:18.543209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 31 14:25:18.543209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application packet.raise_for_error() May 31 14:25:18.545158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 31 14:25:18.545158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 31 14:25:18.545158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 31 14:25:18.545158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 31 14:25:18.545158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 31 14:25:18.545158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 31 14:25:18.545158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [parameters: {'id': 'caddcfaa26954cf793620da9181d88f6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 31 14:25:18.545158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 31 14:25:18.545158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:18.545158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:25:18.545158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:18.545158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:18.545158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:18.545158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:18.545158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:18.545158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:18.547209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:18.547209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:18.547209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:18.547209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:18.547209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:18.547209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:18.547209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 31 14:25:18.547209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 31 14:25:18.547209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:18.547209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:18.547209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 31 14:25:18.547209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 31 14:25:18.547209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 31 14:25:18.547209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 31 14:25:18.547209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 31 14:25:18.547209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:18.551841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 230/477] 10.222.0.67 () {60 vars in 1131 bytes} [Tue May 31 14:25:18 2022] POST /identity/v3/roles => generated 144 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 31 14:25:18.551841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e5064-05f4-4459-ac3c-4810a1d82a8c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:18.570533 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8aa56-1572-4325-9dea-752d16913685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:18.571714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8aa56-1572-4325-9dea-752d16913685 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:18.572312 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8aa56-1572-4325-9dea-752d16913685 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:18.572781 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8aa56-1572-4325-9dea-752d16913685 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:18.573762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8aa56-1572-4325-9dea-752d16913685 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:18.575980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8aa56-1572-4325-9dea-752d16913685 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:18.601871 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 248/478] 10.222.0.67 () {60 vars in 1132 bytes} [Tue May 31 14:25:18 2022] POST /identity/v3/users => generated 249 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:25:18.629218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:18.673309 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e5064-05f4-4459-ac3c-4810a1d82a8c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:18.714277 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1ede6-7626-4d21-a2dd-3f48f211d1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:18.716074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1ede6-7626-4d21-a2dd-3f48f211d1ed None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:18.716582 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1ede6-7626-4d21-a2dd-3f48f211d1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:18.717056 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1ede6-7626-4d21-a2dd-3f48f211d1ed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:18.729640 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f1ede6-7626-4d21-a2dd-3f48f211d1ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:18.738599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-80f1ede6-7626-4d21-a2dd-3f48f211d1ed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:25:18.738599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:18.738599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:18.738599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:18.738599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:18.738599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:18.738599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:18.738599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:18.738599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:18.738599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:18.738599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:18.738599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:18.738599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:18.738599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:18.738599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:18.738599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:18.739557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:18.739557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:18.739557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:18.739557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:18.739557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:18.739557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:18.739557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:18.739557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:18.739557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:18.739557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:18.739557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:18.739557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:18.739557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:18.739557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:18.739557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:18.739557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:18.740391 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:18.740391 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:18.740391 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:18.740391 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:18.740391 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:18.740391 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:18.740391 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:18.740391 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:18.740391 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:18.740391 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:18.740391 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 31 14:25:18.740391 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:18.743331 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 231/479] 10.222.0.67 () {56 vars in 1114 bytes} [Tue May 31 14:25:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:18.751288 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8aa56-1572-4325-9dea-752d16913685 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:18.884924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8aa56-1572-4325-9dea-752d16913685 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:18.925817 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed0555-c784-463f-b990-7eb286c55436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:18.926739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed0555-c784-463f-b990-7eb286c55436 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:18.927030 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed0555-c784-463f-b990-7eb286c55436 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:18.927313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed0555-c784-463f-b990-7eb286c55436 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:18.928075 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed0555-c784-463f-b990-7eb286c55436 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:18.928430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed0555-c784-463f-b990-7eb286c55436 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:18.929879 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed0555-c784-463f-b990-7eb286c55436 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:18.931486 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:18.942804 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 249/480] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:20.307863 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30678f29-3f9a-4232-bf76-561c9ce0d386 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:20.308896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30678f29-3f9a-4232-bf76-561c9ce0d386 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:20.309376 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30678f29-3f9a-4232-bf76-561c9ce0d386 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:20.311286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 232/481] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:20.318743 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f67a70-1710-4426-ab82-fbc7b843fc06 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:20.318743 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f67a70-1710-4426-ab82-fbc7b843fc06 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:20.318743 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f67a70-1710-4426-ab82-fbc7b843fc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:20.343620 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-01f67a70-1710-4426-ab82-fbc7b843fc06 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:20.344908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-01f67a70-1710-4426-ab82-fbc7b843fc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:20.449264 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 250/482] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:20.484368 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30678f29-3f9a-4232-bf76-561c9ce0d386 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:20.606864 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b1a9e-d1b4-47d8-9ca0-ee5994839a7e None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:20.607217 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b1a9e-d1b4-47d8-9ca0-ee5994839a7e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:20.607436 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b1a9e-d1b4-47d8-9ca0-ee5994839a7e None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:20.609102 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 251/483] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:20.613921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a45efc-686f-4794-addc-a8b5ec405a5b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:20.614302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a45efc-686f-4794-addc-a8b5ec405a5b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:20.614495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a45efc-686f-4794-addc-a8b5ec405a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:20.630313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30678f29-3f9a-4232-bf76-561c9ce0d386 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:20.647654 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-39a45efc-686f-4794-addc-a8b5ec405a5b None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:20.648872 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-39a45efc-686f-4794-addc-a8b5ec405a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:20.663063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:20.667097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758b06d-5256-40ce-9e38-5fa0714e7340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:20.668179 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758b06d-5256-40ce-9e38-5fa0714e7340 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:20.668675 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758b06d-5256-40ce-9e38-5fa0714e7340 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:20.669143 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758b06d-5256-40ce-9e38-5fa0714e7340 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:20.688505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b758b06d-5256-40ce-9e38-5fa0714e7340 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:20.697797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-b758b06d-5256-40ce-9e38-5fa0714e7340 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:20.697797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:20.697797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:20.697797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:20.697797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:20.697797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:20.697797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:20.697797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:20.697797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:20.697797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:20.697797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:20.697797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:20.697797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:20.697797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:20.697797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:20.697797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:20.698748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:20.698748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:20.698748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:20.698748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:20.698748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:20.698748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:20.698748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:20.698748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:20.698748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:20.698748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:20.698748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:20.698748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:20.698748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:20.698748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:20.698748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:20.698748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:20.700117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:20.700117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:20.700117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:20.700117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:20.700117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:20.700117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:20.700117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:20.700117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:20.700117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:20.700117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:20.700117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:20.700117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:20.707710 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 233/484] 10.222.0.67 () {56 vars in 1102 bytes} [Tue May 31 14:25:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:20.713461 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758b06d-5256-40ce-9e38-5fa0714e7340 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:20.779968 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 252/485] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:20.803040 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a45efc-686f-4794-addc-a8b5ec405a5b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:20.885896 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:20.908823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758b06d-5256-40ce-9e38-5fa0714e7340 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:20.973525 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a45efc-686f-4794-addc-a8b5ec405a5b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:20.985612 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb701d06-424a-47f8-8293-004889649679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:20.986631 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb701d06-424a-47f8-8293-004889649679 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:20.987134 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb701d06-424a-47f8-8293-004889649679 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:20.987627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb701d06-424a-47f8-8293-004889649679 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:20.988570 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb701d06-424a-47f8-8293-004889649679 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:20.989065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb701d06-424a-47f8-8293-004889649679 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:20.990618 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb701d06-424a-47f8-8293-004889649679 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:21.000622 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 234/486] 10.222.0.67 () {56 vars in 1103 bytes} [Tue May 31 14:25:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:21.005513 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb701d06-424a-47f8-8293-004889649679 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:21.046140 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2443006-6204-499e-b8a1-fa2b02c5f5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:21.047324 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2443006-6204-499e-b8a1-fa2b02c5f5af None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:21.048815 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2443006-6204-499e-b8a1-fa2b02c5f5af None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:21.049286 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2443006-6204-499e-b8a1-fa2b02c5f5af None admin] PATH_INFO: `/v3/roles/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:21.068373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2443006-6204-499e-b8a1-fa2b02c5f5af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:21.087413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-a2443006-6204-499e-b8a1-fa2b02c5f5af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:21.087413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:21.087413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:21.087413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:21.087413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:21.087413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:21.087413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:21.087413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:21.087413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:21.087413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:21.087413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:21.087413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:21.087413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:21.087413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:21.087413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:21.087413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:21.088363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:21.088363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:21.088363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:21.088363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:21.088363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:21.088363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:21.088363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:21.088363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:21.088363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:21.088363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:21.088363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:21.088363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:21.088363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:21.088363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:21.088363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:21.088363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:21.089162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:21.089162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:21.089162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:21.089162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:21.089162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:21.089162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:21.089162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:21.089162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:21.089162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:21.089162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:21.089162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:21.089162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:21.094288 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 253/487] 10.222.0.67 () {56 vars in 1102 bytes} [Tue May 31 14:25:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:25:21.094705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:21.099463 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2443006-6204-499e-b8a1-fa2b02c5f5af None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:21.177451 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb701d06-424a-47f8-8293-004889649679 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:21.215381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f9485-b5c8-492f-8a58-490f245c5234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:21.216189 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f9485-b5c8-492f-8a58-490f245c5234 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:21.216481 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f9485-b5c8-492f-8a58-490f245c5234 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:21.216766 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f9485-b5c8-492f-8a58-490f245c5234 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:21.231147 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f9485-b5c8-492f-8a58-490f245c5234 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:21.232611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f9485-b5c8-492f-8a58-490f245c5234 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:21.253361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-522f9485-b5c8-492f-8a58-490f245c5234 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:21.253361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:21.253361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:21.253361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:21.253361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:21.253361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:21.253361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:21.253361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:21.253361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:21.253361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:21.253361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:21.253361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:21.253361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:21.253361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:21.253361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:21.254289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:21.254289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:21.254289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:21.254289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:21.254289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:21.254289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:21.254289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:21.254289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:21.254289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:21.254289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:21.254289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:21.254289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:21.254289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:21.254289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:21.254289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:21.254289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:21.254289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:21.255635 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:21.255635 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:21.255635 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:21.255635 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:21.255635 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 235/488] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:21.258693 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ea9b8-4319-4d60-9d60-961f4172f978 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:21.258693 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ea9b8-4319-4d60-9d60-961f4172f978 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:21.258693 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ea9b8-4319-4d60-9d60-961f4172f978 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:21.263134 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 236/489] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:21.264155 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ea9b8-4319-4d60-9d60-961f4172f978 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:21.387371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:21.396764 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ea9b8-4319-4d60-9d60-961f4172f978 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:21.403032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2443006-6204-499e-b8a1-fa2b02c5f5af None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:21.422109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:21.444984 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7a90e-26bd-4aed-be4f-063484e43d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:21.445509 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead4763-aef5-48c2-82dc-47c2bb5deb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:21.445852 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7a90e-26bd-4aed-be4f-063484e43d26 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:21.446226 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7a90e-26bd-4aed-be4f-063484e43d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:21.446571 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7a90e-26bd-4aed-be4f-063484e43d26 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:21.446910 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead4763-aef5-48c2-82dc-47c2bb5deb24 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:21.447344 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead4763-aef5-48c2-82dc-47c2bb5deb24 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:21.447561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7a90e-26bd-4aed-be4f-063484e43d26 None admin] RBAC: Adding query filter params (name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:21.447928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead4763-aef5-48c2-82dc-47c2bb5deb24 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:21.448036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7a90e-26bd-4aed-be4f-063484e43d26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:21.448908 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ead4763-aef5-48c2-82dc-47c2bb5deb24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:21.449418 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ead4763-aef5-48c2-82dc-47c2bb5deb24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:21.449642 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7a90e-26bd-4aed-be4f-063484e43d26 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:21.451783 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ead4763-aef5-48c2-82dc-47c2bb5deb24 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:21.464922 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 254/490] 10.222.0.67 () {56 vars in 1103 bytes} [Tue May 31 14:25:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:21.464922 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7847f0-fb6c-4db2-b9a1-1ec57fca61d3 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:21.464922 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7847f0-fb6c-4db2-b9a1-1ec57fca61d3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:21.464922 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7847f0-fb6c-4db2-b9a1-1ec57fca61d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:21.474000 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 237/491] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:21.475268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead4763-aef5-48c2-82dc-47c2bb5deb24 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:21.482828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-4a7847f0-fb6c-4db2-b9a1-1ec57fca61d3 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:21.484088 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4a7847f0-fb6c-4db2-b9a1-1ec57fca61d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:21.562924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 255/492] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:21.563978 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7847f0-fb6c-4db2-b9a1-1ec57fca61d3 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:21.581574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead4763-aef5-48c2-82dc-47c2bb5deb24 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:21.619121 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb8358-991a-4785-af87-e8481cd03905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:21.620169 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb8358-991a-4785-af87-e8481cd03905 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:21.620460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb8358-991a-4785-af87-e8481cd03905 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:21.620741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb8358-991a-4785-af87-e8481cd03905 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:21.631315 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb8358-991a-4785-af87-e8481cd03905 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:21.632753 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb8358-991a-4785-af87-e8481cd03905 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:21.650819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-8bbb8358-991a-4785-af87-e8481cd03905 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:21.650819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:21.650819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:21.650819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:21.650819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:21.650819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:21.650819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:21.650819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:21.650819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:21.650819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:21.650819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:21.650819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:21.650819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:21.650819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:21.650819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:21.651803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:21.651803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:21.651803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:21.651803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:21.651803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:21.651803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:21.651803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:21.651803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:21.651803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:21.651803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:21.651803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:21.651803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:21.651803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:21.651803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:21.651803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:21.651803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:21.651803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:21.653251 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:21.653251 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:21.653251 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:21.653251 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:21.653251 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 238/493] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:21.653984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880fc5a-166d-453d-9406-8d87b77a700a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:21.654082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880fc5a-166d-453d-9406-8d87b77a700a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:21.654338 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880fc5a-166d-453d-9406-8d87b77a700a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:21.656105 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 239/494] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:21.656819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880fc5a-166d-453d-9406-8d87b77a700a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:21.675258 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:21.691630 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7847f0-fb6c-4db2-b9a1-1ec57fca61d3 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:21.705713 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:21.732703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db533c-8c75-4206-a3aa-b34bfa769fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:21.733673 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db533c-8c75-4206-a3aa-b34bfa769fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:21.734019 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db533c-8c75-4206-a3aa-b34bfa769fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:21.734302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db533c-8c75-4206-a3aa-b34bfa769fb9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:21.742305 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db533c-8c75-4206-a3aa-b34bfa769fb9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:21.743802 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db533c-8c75-4206-a3aa-b34bfa769fb9 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:21.761228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-56db533c-8c75-4206-a3aa-b34bfa769fb9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 31 14:25:21.761228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:21.761228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:21.761228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:21.761228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:21.761228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:21.761228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:21.761228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:21.761228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:21.761228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:21.761228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:21.761228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:21.761228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:25:21.761228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:25:21.761228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:25:21.761228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:25:21.762152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:21.762152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:21.762152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:25:21.762152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:21.762152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:25:21.762152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:21.762152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:21.762152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:21.762152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:21.762152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:21.762152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:21.762152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:21.762152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:21.762152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:21.762152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:21.762152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:21.762152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:21.762152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:21.763462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:21.763462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:21.763462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:25:21.763462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:25:21.763462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:25:21.763462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:25:21.763462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:25:21.763462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:25:21.763462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 31 14:25:21.763462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:21.763462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 256/495] 10.222.0.67 () {56 vars in 1093 bytes} [Tue May 31 14:25:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:21.764029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a91c5c-bbb1-4648-b222-5541206acdd8 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:21.764411 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a91c5c-bbb1-4648-b222-5541206acdd8 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:21.764411 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a91c5c-bbb1-4648-b222-5541206acdd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:21.778725 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880fc5a-166d-453d-9406-8d87b77a700a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:21.781438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-d2a91c5c-bbb1-4648-b222-5541206acdd8 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:21.782638 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d2a91c5c-bbb1-4648-b222-5541206acdd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:21.813204 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615ea10-0cd1-43ba-8402-0f918ed54f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:21.813991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615ea10-0cd1-43ba-8402-0f918ed54f7b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:21.814279 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615ea10-0cd1-43ba-8402-0f918ed54f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:21.814555 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615ea10-0cd1-43ba-8402-0f918ed54f7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:21.824822 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3615ea10-0cd1-43ba-8402-0f918ed54f7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:21.832735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-3615ea10-0cd1-43ba-8402-0f918ed54f7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:21.832735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:21.832735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:21.832735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:21.832735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:21.832735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:21.832735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:21.832735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:21.832735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:21.832735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:21.832735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:21.832735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:21.832735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:21.832735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:21.832735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:21.832735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:21.833633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:21.833633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:21.833633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:21.833633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:21.833633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:21.833633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:21.833633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:21.833633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:21.833633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:21.833633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:21.833633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:21.833633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:21.833633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:21.833633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:21.833633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:21.833633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:21.834422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:21.834422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:21.834422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:21.834422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:21.834422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:21.834422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:21.834422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:21.834422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:21.834422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:21.834422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:21.834422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:21.834422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:21.838600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 240/496] 10.222.0.67 () {56 vars in 1102 bytes} [Tue May 31 14:25:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:21.839381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615ea10-0cd1-43ba-8402-0f918ed54f7b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:21.848384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 257/497] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:21.849108 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a91c5c-bbb1-4648-b222-5541206acdd8 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:21.908675 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:21.942245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615ea10-0cd1-43ba-8402-0f918ed54f7b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:21.946412 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:21.950413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a91c5c-bbb1-4648-b222-5541206acdd8 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:21.975488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3b923-6ba0-4e50-b7c2-4cdabfb08ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:21.976150 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3b923-6ba0-4e50-b7c2-4cdabfb08ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:21.976437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3b923-6ba0-4e50-b7c2-4cdabfb08ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:21.976715 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3b923-6ba0-4e50-b7c2-4cdabfb08ea7 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:21.977432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3b923-6ba0-4e50-b7c2-4cdabfb08ea7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:21.977742 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3b923-6ba0-4e50-b7c2-4cdabfb08ea7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:21.979150 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3b923-6ba0-4e50-b7c2-4cdabfb08ea7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:21.985131 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1b692-770e-4f6d-9997-d8479aa42443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:21.985744 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1b692-770e-4f6d-9997-d8479aa42443 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:21.986031 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1b692-770e-4f6d-9997-d8479aa42443 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:21.986789 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1b692-770e-4f6d-9997-d8479aa42443 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:21.987643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1b692-770e-4f6d-9997-d8479aa42443 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:21.987959 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1b692-770e-4f6d-9997-d8479aa42443 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:21.989852 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1b692-770e-4f6d-9997-d8479aa42443 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:21.991350 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 241/498] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:21.992389 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e306bf-2b14-4e63-8d67-8d20c88ae1e7 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:21.992676 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e306bf-2b14-4e63-8d67-8d20c88ae1e7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:21.992951 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e306bf-2b14-4e63-8d67-8d20c88ae1e7 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:21.994290 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 242/499] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:21.994876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e306bf-2b14-4e63-8d67-8d20c88ae1e7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:22.033358 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 258/500] 10.222.0.67 () {56 vars in 1133 bytes} [Tue May 31 14:25:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:22.034800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1b692-770e-4f6d-9997-d8479aa42443 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:22.101723 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e306bf-2b14-4e63-8d67-8d20c88ae1e7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:22.135779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1b692-770e-4f6d-9997-d8479aa42443 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:22.172020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db15c4-4195-4a9e-aae3-9d8cf1214ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:22.172837 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db15c4-4195-4a9e-aae3-9d8cf1214ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:22.173121 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db15c4-4195-4a9e-aae3-9d8cf1214ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:22.173393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db15c4-4195-4a9e-aae3-9d8cf1214ba6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:22.184966 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db15c4-4195-4a9e-aae3-9d8cf1214ba6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:22.192977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-66db15c4-4195-4a9e-aae3-9d8cf1214ba6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:22.192977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:22.192977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:22.192977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:22.192977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:22.192977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:22.192977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:22.192977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:22.192977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:22.192977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:22.192977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:22.192977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:22.192977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:22.192977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:22.192977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:22.192977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:22.194111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:22.194111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:22.194111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:22.194111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:22.194111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:22.194111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:22.194111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:22.194111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:22.194111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:22.194111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:22.194111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:22.194111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:22.194111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:22.194111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:22.194111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:22.194111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:22.195071 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:22.195071 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:22.195071 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:22.195071 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:22.195071 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:22.195071 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:22.195071 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:22.195071 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:22.195071 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:22.195071 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:22.195071 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:22.195071 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:22.199503 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 259/501] 10.222.0.67 () {56 vars in 1102 bytes} [Tue May 31 14:25:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:22.201375 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca07011-9172-42f3-bef5-410c54a24d94 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:22.201931 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca07011-9172-42f3-bef5-410c54a24d94 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:22.202214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca07011-9172-42f3-bef5-410c54a24d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:22.220143 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6ca07011-9172-42f3-bef5-410c54a24d94 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:22.221370 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6ca07011-9172-42f3-bef5-410c54a24d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:22.232686 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:22.249603 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:22.258297 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f0425-894f-40ee-b297-25691a492866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:22.259000 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f0425-894f-40ee-b297-25691a492866 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:22.259290 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f0425-894f-40ee-b297-25691a492866 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:22.259568 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f0425-894f-40ee-b297-25691a492866 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:22.260323 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f0425-894f-40ee-b297-25691a492866 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:22.260635 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f0425-894f-40ee-b297-25691a492866 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:22.262543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f0425-894f-40ee-b297-25691a492866 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:22.270580 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 243/502] 10.222.0.67 () {56 vars in 1103 bytes} [Tue May 31 14:25:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:22.271384 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f0425-894f-40ee-b297-25691a492866 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:22.290423 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 260/503] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:22.291280 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca07011-9172-42f3-bef5-410c54a24d94 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:22.370020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f0425-894f-40ee-b297-25691a492866 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:22.396444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca07011-9172-42f3-bef5-410c54a24d94 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:22.404352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f45e4-18de-443c-9e55-56a2cae3639b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:22.405153 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f45e4-18de-443c-9e55-56a2cae3639b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:22.405441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f45e4-18de-443c-9e55-56a2cae3639b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:22.405769 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f45e4-18de-443c-9e55-56a2cae3639b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:22.412655 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f45e4-18de-443c-9e55-56a2cae3639b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:22.414078 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f45e4-18de-443c-9e55-56a2cae3639b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:22.428533 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8480b-8f5f-4eaf-aecb-dea95b96f9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:22.429174 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8480b-8f5f-4eaf-aecb-dea95b96f9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:22.429452 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8480b-8f5f-4eaf-aecb-dea95b96f9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:22.429544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-629f45e4-18de-443c-9e55-56a2cae3639b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 31 14:25:22.429544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:22.429544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:22.429544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:22.429544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:22.429544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:22.429544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:22.429544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:22.429544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:22.429544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:22.429544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:22.429544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:22.429544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:25:22.429544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:25:22.429544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:25:22.429544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:25:22.430551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:22.430551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:22.430551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:25:22.430551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:22.430551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:25:22.430551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:22.430551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:22.430551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:22.430551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:22.430551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:22.430551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:22.430551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:22.430551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:22.430551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:22.430551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:22.430551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:22.430551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:22.430551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:22.431506 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:22.431506 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:22.431506 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:25:22.431506 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:25:22.431506 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:25:22.431506 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:25:22.431506 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:25:22.431506 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:25:22.431506 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 31 14:25:22.431506 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:22.432018 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8480b-8f5f-4eaf-aecb-dea95b96f9d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:22.432086 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 244/504] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:25:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:22.432086 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f45e4-18de-443c-9e55-56a2cae3639b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:22.447521 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:22.480272 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:22.499677 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f8480b-8f5f-4eaf-aecb-dea95b96f9d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:22.501069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f8480b-8f5f-4eaf-aecb-dea95b96f9d7 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:22.529794 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f45e4-18de-443c-9e55-56a2cae3639b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:22.562247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3407987-1c0b-45d1-bd43-a2e24b874fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:22.562935 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3407987-1c0b-45d1-bd43-a2e24b874fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:22.563216 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3407987-1c0b-45d1-bd43-a2e24b874fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:22.563490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3407987-1c0b-45d1-bd43-a2e24b874fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:22.564229 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3407987-1c0b-45d1-bd43-a2e24b874fa7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:22.564533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3407987-1c0b-45d1-bd43-a2e24b874fa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:22.565878 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3407987-1c0b-45d1-bd43-a2e24b874fa7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:22.573533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 245/505] 10.222.0.67 () {56 vars in 1103 bytes} [Tue May 31 14:25:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:22.574637 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3407987-1c0b-45d1-bd43-a2e24b874fa7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:22.598399 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 261/506] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:25:22.599275 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8480b-8f5f-4eaf-aecb-dea95b96f9d7 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:22.645389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:22.679070 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3407987-1c0b-45d1-bd43-a2e24b874fa7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:22.691934 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:22.698212 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8480b-8f5f-4eaf-aecb-dea95b96f9d7 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:22.713433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e29c37-25f1-4324-b65f-66193449143a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:22.714225 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e29c37-25f1-4324-b65f-66193449143a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:22.714507 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e29c37-25f1-4324-b65f-66193449143a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:22.715303 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e29c37-25f1-4324-b65f-66193449143a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:22.725406 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e29c37-25f1-4324-b65f-66193449143a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:22.726865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e29c37-25f1-4324-b65f-66193449143a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:22.731820 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e1f50-b23a-4fa0-8bbb-3ad6ed2e3e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:22.732610 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e1f50-b23a-4fa0-8bbb-3ad6ed2e3e4d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:22.732893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e1f50-b23a-4fa0-8bbb-3ad6ed2e3e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:22.733171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e1f50-b23a-4fa0-8bbb-3ad6ed2e3e4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:22.742864 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e1f50-b23a-4fa0-8bbb-3ad6ed2e3e4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:22.743918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-90e29c37-25f1-4324-b65f-66193449143a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:22.743918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:22.743918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:22.743918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:22.743918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:22.743918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:22.743918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:22.743918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:22.743918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:22.743918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:22.743918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:22.743918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:22.743918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:22.743918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:22.743918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:22.744841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:22.744841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:22.744841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:22.744841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:22.744841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:22.744841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:22.744841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:22.744841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:22.744841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:22.744841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:22.744841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:22.744841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:22.744841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:22.744841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:22.744841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:22.744841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:22.744841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:22.745665 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:22.745665 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:22.745665 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:22.745665 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:22.745883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e1f50-b23a-4fa0-8bbb-3ad6ed2e3e4d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:22.745951 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 246/507] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:22.745951 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e29c37-25f1-4324-b65f-66193449143a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:22.766907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-9d0e1f50-b23a-4fa0-8bbb-3ad6ed2e3e4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:22.766907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:22.766907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:22.766907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:22.766907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:22.766907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:22.766907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:22.766907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:22.766907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:22.766907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:22.766907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:22.766907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:22.766907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:22.766907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:22.766907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:22.767918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:22.767918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:22.767918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:22.767918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:22.767918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:22.767918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:22.767918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:22.767918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:22.767918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:22.767918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:22.767918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:22.767918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:22.767918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:22.767918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:22.767918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:22.767918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:22.767918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:22.768778 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:22.768778 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:22.768778 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:22.768778 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:22.768778 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 262/508] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:22.768778 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e1f50-b23a-4fa0-8bbb-3ad6ed2e3e4d None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:22.849154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e29c37-25f1-4324-b65f-66193449143a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:22.863191 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:22.866854 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e1f50-b23a-4fa0-8bbb-3ad6ed2e3e4d None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:22.882912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14a99d-7342-4f06-b130-6f6332f29d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:22.883588 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14a99d-7342-4f06-b130-6f6332f29d01 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:22.883876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14a99d-7342-4f06-b130-6f6332f29d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:22.884162 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14a99d-7342-4f06-b130-6f6332f29d01 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:22.884995 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14a99d-7342-4f06-b130-6f6332f29d01 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:22.885307 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14a99d-7342-4f06-b130-6f6332f29d01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:22.887303 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14a99d-7342-4f06-b130-6f6332f29d01 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:22.900987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce92caf-403b-43d5-8356-435a2a09226c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:22.901777 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce92caf-403b-43d5-8356-435a2a09226c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:22.902059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce92caf-403b-43d5-8356-435a2a09226c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:22.902965 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce92caf-403b-43d5-8356-435a2a09226c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:22.912769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce92caf-403b-43d5-8356-435a2a09226c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:22.914249 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce92caf-403b-43d5-8356-435a2a09226c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:22.926161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 247/509] 10.222.0.67 () {56 vars in 1137 bytes} [Tue May 31 14:25:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:22.927011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14a99d-7342-4f06-b130-6f6332f29d01 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:22.931351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-0ce92caf-403b-43d5-8356-435a2a09226c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:22.931351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:22.931351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:22.931351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:22.931351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:22.931351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:22.931351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:22.931351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:22.931351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:22.931351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:22.931351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:22.931351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:22.931351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:22.931351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:22.931351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:22.932344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:22.932344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:22.932344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:22.932344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:22.932344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:22.932344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:22.932344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:22.932344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:22.932344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:22.932344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:22.932344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:22.932344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:22.932344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:22.932344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:22.932344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:22.932344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:22.932344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:22.933228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:22.933228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:22.933228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:22.933228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:22.933228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 263/510] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:22.933892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce92caf-403b-43d5-8356-435a2a09226c None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:22.955214 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:23.031717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14a99d-7342-4f06-b130-6f6332f29d01 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:23.035463 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce92caf-403b-43d5-8356-435a2a09226c None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:23.064352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44877b93-85aa-48f6-b520-72a30ea83691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:23.065145 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44877b93-85aa-48f6-b520-72a30ea83691 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:23.065429 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44877b93-85aa-48f6-b520-72a30ea83691 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:23.065709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44877b93-85aa-48f6-b520-72a30ea83691 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:23.067905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a82cb4-c5e2-4d52-b4a4-90d1120f2fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:23.068561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a82cb4-c5e2-4d52-b4a4-90d1120f2fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:23.068846 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a82cb4-c5e2-4d52-b4a4-90d1120f2fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:23.069123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a82cb4-c5e2-4d52-b4a4-90d1120f2fc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:23.069846 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a82cb4-c5e2-4d52-b4a4-90d1120f2fc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:23.070156 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a82cb4-c5e2-4d52-b4a4-90d1120f2fc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:23.071560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a82cb4-c5e2-4d52-b4a4-90d1120f2fc0 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:23.075937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44877b93-85aa-48f6-b520-72a30ea83691 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:23.077635 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44877b93-85aa-48f6-b520-72a30ea83691 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:23.083310 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 264/511] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:23.084132 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a82cb4-c5e2-4d52-b4a4-90d1120f2fc0 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:23.096558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:23.099574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-44877b93-85aa-48f6-b520-72a30ea83691 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:23.099574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:23.099574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:23.099574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:23.099574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:23.099574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:23.099574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:23.099574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:23.099574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:23.099574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:23.099574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:23.099574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:23.099574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:23.099574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:23.099574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:23.100421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:23.100421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:23.100421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:23.100421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:23.100421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:23.100421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:23.100421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:23.100421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:23.100421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:23.100421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:23.100421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:23.100421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:23.100421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:23.100421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:23.100421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:23.100421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:23.100421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:23.101260 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:23.101260 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:23.101260 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:23.101260 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:23.101260 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 248/512] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:23.101866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44877b93-85aa-48f6-b520-72a30ea83691 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:23.177358 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:23.187678 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a82cb4-c5e2-4d52-b4a4-90d1120f2fc0 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:23.203023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44877b93-85aa-48f6-b520-72a30ea83691 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:23.222210 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7882e-6f79-41f9-92a1-9402cc912f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:23.222898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7882e-6f79-41f9-92a1-9402cc912f86 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:23.223194 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7882e-6f79-41f9-92a1-9402cc912f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:23.223478 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7882e-6f79-41f9-92a1-9402cc912f86 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:23.224197 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7882e-6f79-41f9-92a1-9402cc912f86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:23.224508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7882e-6f79-41f9-92a1-9402cc912f86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:23.225881 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7882e-6f79-41f9-92a1-9402cc912f86 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:23.236581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4aa009-62fc-4539-a948-24c2de5017d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:23.237151 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 265/513] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:23.237241 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4aa009-62fc-4539-a948-24c2de5017d6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:23.237505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4aa009-62fc-4539-a948-24c2de5017d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:23.237785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4aa009-62fc-4539-a948-24c2de5017d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:23.237957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7882e-6f79-41f9-92a1-9402cc912f86 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:23.303936 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4aa009-62fc-4539-a948-24c2de5017d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:23.305805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4aa009-62fc-4539-a948-24c2de5017d6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:23.306938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:23.339008 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7882e-6f79-41f9-92a1-9402cc912f86 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:23.371756 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faebe8d-e102-4723-8569-22027adf147c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:23.372410 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faebe8d-e102-4723-8569-22027adf147c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:23.372699 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faebe8d-e102-4723-8569-22027adf147c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:23.372973 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faebe8d-e102-4723-8569-22027adf147c None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:23.373695 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faebe8d-e102-4723-8569-22027adf147c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:23.374004 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faebe8d-e102-4723-8569-22027adf147c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:23.375472 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faebe8d-e102-4723-8569-22027adf147c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:23.378322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:23.387281 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 266/514] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:23.388075 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faebe8d-e102-4723-8569-22027adf147c None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:23.407797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 249/515] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:25:23.408618 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4aa009-62fc-4539-a948-24c2de5017d6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:23.487815 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faebe8d-e102-4723-8569-22027adf147c None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:23.507173 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4aa009-62fc-4539-a948-24c2de5017d6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:23.514863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:23.520991 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf803dbf-135d-46e2-9ab2-b23aa44a20b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:23.521778 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf803dbf-135d-46e2-9ab2-b23aa44a20b3 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:23.522361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf803dbf-135d-46e2-9ab2-b23aa44a20b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:23.522361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf803dbf-135d-46e2-9ab2-b23aa44a20b3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:23.529808 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf803dbf-135d-46e2-9ab2-b23aa44a20b3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:23.531305 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf803dbf-135d-46e2-9ab2-b23aa44a20b3 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:23.541145 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd7ce47-345a-46a3-9908-1243e394958d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:23.541802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7ce47-345a-46a3-9908-1243e394958d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:23.542088 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7ce47-345a-46a3-9908-1243e394958d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:23.542365 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7ce47-345a-46a3-9908-1243e394958d None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:23.543630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd7ce47-345a-46a3-9908-1243e394958d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:23.543939 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd7ce47-345a-46a3-9908-1243e394958d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:23.545285 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd7ce47-345a-46a3-9908-1243e394958d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:23.545417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-cf803dbf-135d-46e2-9ab2-b23aa44a20b3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 31 14:25:23.545417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:23.545417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:23.545417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:23.545417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:23.545417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:23.545417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:23.545417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:23.545417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:23.545417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:23.545417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:23.545417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:23.545417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:25:23.545417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:25:23.545417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:25:23.545417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:25:23.546441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:23.546441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:23.546441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:25:23.546441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:23.546441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:25:23.546441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:23.546441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:23.546441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:23.546441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:23.546441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:23.546441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:23.546441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:23.546441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:23.546441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:23.546441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:23.546441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:23.546441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:23.546441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:23.547940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:23.547940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:23.547940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:25:23.547940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:25:23.547940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:25:23.547940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:25:23.547940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:25:23.547940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:25:23.547940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 31 14:25:23.547940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:23.547940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 267/516] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:25:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:23.547940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf803dbf-135d-46e2-9ab2-b23aa44a20b3 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:23.559801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 250/517] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:23.560568 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd7ce47-345a-46a3-9908-1243e394958d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:23.597343 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:23.645711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf803dbf-135d-46e2-9ab2-b23aa44a20b3 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:23.663782 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd7ce47-345a-46a3-9908-1243e394958d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:23.678565 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf486332-1290-4a78-b08d-e103ebe358ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:23.679259 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf486332-1290-4a78-b08d-e103ebe358ad None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:23.679542 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf486332-1290-4a78-b08d-e103ebe358ad None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:23.679825 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf486332-1290-4a78-b08d-e103ebe358ad None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:23.680522 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf486332-1290-4a78-b08d-e103ebe358ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:23.682040 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf486332-1290-4a78-b08d-e103ebe358ad None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:23.696561 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f61d51-fef3-4f83-916c-67f0f0ee50b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:23.697736 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f61d51-fef3-4f83-916c-67f0f0ee50b0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:23.698030 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f61d51-fef3-4f83-916c-67f0f0ee50b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:23.698310 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f61d51-fef3-4f83-916c-67f0f0ee50b0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:23.701916 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 268/518] 10.222.0.67 () {60 vars in 1132 bytes} [Tue May 31 14:25:23 2022] POST /identity/v3/users => generated 247 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:25:23.702723 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf486332-1290-4a78-b08d-e103ebe358ad None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:23.707175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f61d51-fef3-4f83-916c-67f0f0ee50b0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:23.708647 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f61d51-fef3-4f83-916c-67f0f0ee50b0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:23.728705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-c9f61d51-fef3-4f83-916c-67f0f0ee50b0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 31 14:25:23.728705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:23.728705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:23.728705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:23.728705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:23.728705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:23.728705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:23.728705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:23.728705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:23.728705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:23.728705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:23.728705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:23.728705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:25:23.728705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:25:23.728705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:25:23.728705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:25:23.729708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:23.729708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:23.729708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:25:23.729708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:23.729708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:25:23.729708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:23.729708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:23.729708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:23.729708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:23.729708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:23.729708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:23.729708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:23.729708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:23.729708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:23.729708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:23.729708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:23.729708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:23.729708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:23.736153 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:23.736153 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:23.736153 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:25:23.736153 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:25:23.736153 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:25:23.736153 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:25:23.736153 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:25:23.736153 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:25:23.736153 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 31 14:25:23.736153 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:23.736153 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 251/519] 10.222.0.67 () {56 vars in 1102 bytes} [Tue May 31 14:25:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:23.736153 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f61d51-fef3-4f83-916c-67f0f0ee50b0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:23.767157 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:23.829736 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf486332-1290-4a78-b08d-e103ebe358ad None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:23.846808 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:23.855706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f61d51-fef3-4f83-916c-67f0f0ee50b0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:23.866848 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a340a42-4022-497e-8bbe-ced4eac86b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:23.867761 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a340a42-4022-497e-8bbe-ced4eac86b71 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:23.868114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a340a42-4022-497e-8bbe-ced4eac86b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:23.868394 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a340a42-4022-497e-8bbe-ced4eac86b71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:23.880647 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a340a42-4022-497e-8bbe-ced4eac86b71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:23.882161 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a340a42-4022-497e-8bbe-ced4eac86b71 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:23.893466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad0314-e612-4253-98fa-890dc2f3cbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:23.894417 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad0314-e612-4253-98fa-890dc2f3cbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:23.894859 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad0314-e612-4253-98fa-890dc2f3cbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:23.895256 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad0314-e612-4253-98fa-890dc2f3cbe1 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:23.897208 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad0314-e612-4253-98fa-890dc2f3cbe1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:23.897208 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad0314-e612-4253-98fa-890dc2f3cbe1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:23.898410 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad0314-e612-4253-98fa-890dc2f3cbe1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:23.901392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-6a340a42-4022-497e-8bbe-ced4eac86b71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:23.901392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:23.901392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:23.901392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:23.901392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:23.901392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:23.901392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:23.901392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:23.901392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:23.901392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:23.901392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:23.901392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:23.901392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:23.901392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:23.901392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:23.902379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:23.902379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:23.902379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:23.902379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:23.902379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:23.902379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:23.902379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:23.902379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:23.902379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:23.902379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:23.902379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:23.902379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:23.902379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:23.902379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:23.902379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:23.902379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:23.902379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:23.903377 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:23.903377 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:23.903377 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:23.903377 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:23.903377 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 269/520] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:23.903636 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a340a42-4022-497e-8bbe-ced4eac86b71 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:23.919317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 252/521] 10.222.0.67 () {56 vars in 1137 bytes} [Tue May 31 14:25:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:23.924023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad0314-e612-4253-98fa-890dc2f3cbe1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:24.015666 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:24.020556 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a340a42-4022-497e-8bbe-ced4eac86b71 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:24.042839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad0314-e612-4253-98fa-890dc2f3cbe1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:24.063284 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757524f0-dae1-4eaf-a0c4-92b0dca9fb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:24.064084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757524f0-dae1-4eaf-a0c4-92b0dca9fb65 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:24.064366 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757524f0-dae1-4eaf-a0c4-92b0dca9fb65 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:24.064642 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757524f0-dae1-4eaf-a0c4-92b0dca9fb65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:24.071467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757524f0-dae1-4eaf-a0c4-92b0dca9fb65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:24.072914 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757524f0-dae1-4eaf-a0c4-92b0dca9fb65 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:24.080620 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e0243-d4f6-401f-b992-f55417d48ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:24.081246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e0243-d4f6-401f-b992-f55417d48ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:24.081550 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e0243-d4f6-401f-b992-f55417d48ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:24.081817 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e0243-d4f6-401f-b992-f55417d48ac9 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:24.082683 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e0243-d4f6-401f-b992-f55417d48ac9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:24.083066 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e0243-d4f6-401f-b992-f55417d48ac9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:24.084499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e0243-d4f6-401f-b992-f55417d48ac9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:24.088703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-757524f0-dae1-4eaf-a0c4-92b0dca9fb65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:24.088703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:24.088703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:24.088703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:24.088703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:24.088703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:24.088703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:24.088703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:24.088703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:24.088703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:24.088703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:24.088703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:24.088703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:25:24.088703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:25:24.088703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:25:24.089548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:25:24.089548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:24.089548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:24.089548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:24.089548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:24.089548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:24.089548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:24.089548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:24.089548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:24.089548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:24.089548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:24.089548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:24.089548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:24.089548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:24.089548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:24.089548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:24.089548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:24.090403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:25:24.090403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:25:24.090403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:25:24.090403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:25:24.090403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:25:24.090403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:25:24.090403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:24.090403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:24.094687 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 270/522] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:24.095491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757524f0-dae1-4eaf-a0c4-92b0dca9fb65 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:24.107839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 253/523] 10.222.0.67 () {56 vars in 1139 bytes} [Tue May 31 14:25:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:24.107839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e0243-d4f6-401f-b992-f55417d48ac9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:24.119573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:24.211188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757524f0-dae1-4eaf-a0c4-92b0dca9fb65 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:24.214179 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e0243-d4f6-401f-b992-f55417d48ac9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:24.247765 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856fa18-0f01-4014-b926-dad8de8202a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:24.248434 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856fa18-0f01-4014-b926-dad8de8202a0 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:24.248718 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b9232-59cf-4dee-9bc0-0231d50e97da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:24.248831 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856fa18-0f01-4014-b926-dad8de8202a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:24.248987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856fa18-0f01-4014-b926-dad8de8202a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:24.249376 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b9232-59cf-4dee-9bc0-0231d50e97da None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:24.249666 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b9232-59cf-4dee-9bc0-0231d50e97da None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:24.249764 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856fa18-0f01-4014-b926-dad8de8202a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:24.249950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b9232-59cf-4dee-9bc0-0231d50e97da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:24.250041 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856fa18-0f01-4014-b926-dad8de8202a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:24.251499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856fa18-0f01-4014-b926-dad8de8202a0 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:24.267745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 271/524] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:24.268406 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856fa18-0f01-4014-b926-dad8de8202a0 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:24.287870 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:24.331932 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b9232-59cf-4dee-9bc0-0231d50e97da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:24.333191 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b9232-59cf-4dee-9bc0-0231d50e97da None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:24.346932 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:24.380470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856fa18-0f01-4014-b926-dad8de8202a0 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:24.416241 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794766ae-a4f3-4b1b-9960-f3c9fc2c44cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:24.416925 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794766ae-a4f3-4b1b-9960-f3c9fc2c44cd None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:24.417213 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794766ae-a4f3-4b1b-9960-f3c9fc2c44cd None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:24.417478 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794766ae-a4f3-4b1b-9960-f3c9fc2c44cd None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:24.418793 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794766ae-a4f3-4b1b-9960-f3c9fc2c44cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:24.419198 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794766ae-a4f3-4b1b-9960-f3c9fc2c44cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:24.420525 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794766ae-a4f3-4b1b-9960-f3c9fc2c44cd None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:24.444916 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 272/525] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:25:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:24.444916 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794766ae-a4f3-4b1b-9960-f3c9fc2c44cd None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:24.462846 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 254/526] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:25:24.462846 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b9232-59cf-4dee-9bc0-0231d50e97da None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:24.532730 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:24.557586 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794766ae-a4f3-4b1b-9960-f3c9fc2c44cd None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:24.560526 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b9232-59cf-4dee-9bc0-0231d50e97da None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:24.572082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:24.596347 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e48fb-cf5a-4e2d-a440-c8ee56ba3117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:24.596927 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e48fb-cf5a-4e2d-a440-c8ee56ba3117 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:24.597215 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e48fb-cf5a-4e2d-a440-c8ee56ba3117 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:24.597494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e48fb-cf5a-4e2d-a440-c8ee56ba3117 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:24.598020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e0088-9b94-45ad-81e3-da5dcb3959ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:24.598844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e0088-9b94-45ad-81e3-da5dcb3959ab None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:24.599128 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e0088-9b94-45ad-81e3-da5dcb3959ab None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:24.599396 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e0088-9b94-45ad-81e3-da5dcb3959ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:24.606666 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e0088-9b94-45ad-81e3-da5dcb3959ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:24.608110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e0088-9b94-45ad-81e3-da5dcb3959ab None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:24.622621 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-ea6e0088-9b94-45ad-81e3-da5dcb3959ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:24.622621 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:24.622621 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:24.622621 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:24.622621 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:24.622621 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:24.622621 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:24.622621 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:24.622621 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:24.622621 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:24.622621 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:24.622621 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:24.622621 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:25:24.622621 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:25:24.622621 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:25:24.623673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:25:24.623673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:24.623673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:24.623673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:24.623673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:24.623673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:24.623673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:24.623673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:24.623673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:24.623673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:24.623673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:24.623673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:24.623673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:24.623673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:24.623673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:24.623673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:24.623673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:24.624536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:25:24.624536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:25:24.624536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:25:24.624536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:25:24.624536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:25:24.624536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:25:24.624536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:24.624536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:24.624536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 255/527] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:24.625006 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e0088-9b94-45ad-81e3-da5dcb3959ab None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:24.682945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e48fb-cf5a-4e2d-a440-c8ee56ba3117 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:24.684651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e48fb-cf5a-4e2d-a440-c8ee56ba3117 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:24.753606 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e0088-9b94-45ad-81e3-da5dcb3959ab None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:24.781269 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:24.788784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d170e5-4afd-484a-a65d-c47bdc77921f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:24.789409 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d170e5-4afd-484a-a65d-c47bdc77921f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:24.789698 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d170e5-4afd-484a-a65d-c47bdc77921f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:24.789979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d170e5-4afd-484a-a65d-c47bdc77921f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:24.790850 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d170e5-4afd-484a-a65d-c47bdc77921f None admin] RBAC: Adding query filter params (role.id=5956aa478bfc4d03919395e637e61c83, scope.project.id=f965a93a2f674598890bd8cafededb2c, user.id=68c6363b60954c3ebf50d2019f5f2fd2) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:24.791173 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d170e5-4afd-484a-a65d-c47bdc77921f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:24.792613 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d170e5-4afd-484a-a65d-c47bdc77921f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:24.806689 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 256/528] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:25:24 2022] GET /identity/v3/role_assignments?user.id=68c6363b60954c3ebf50d2019f5f2fd2&scope.project.id=f965a93a2f674598890bd8cafededb2c&role.id=5956aa478bfc4d03919395e637e61c83 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:24.806689 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d170e5-4afd-484a-a65d-c47bdc77921f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:24.819737 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 273/529] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:25:24.820953 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e48fb-cf5a-4e2d-a440-c8ee56ba3117 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:24.848191 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:24.950104 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d170e5-4afd-484a-a65d-c47bdc77921f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:24.961285 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e48fb-cf5a-4e2d-a440-c8ee56ba3117 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:25.001141 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffad3f0-1c0c-46f2-a8cf-222e2e11d660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:25.001816 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffad3f0-1c0c-46f2-a8cf-222e2e11d660 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:25.002099 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffad3f0-1c0c-46f2-a8cf-222e2e11d660 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:25.002381 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffad3f0-1c0c-46f2-a8cf-222e2e11d660 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:25.003217 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffad3f0-1c0c-46f2-a8cf-222e2e11d660 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:25.003530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffad3f0-1c0c-46f2-a8cf-222e2e11d660 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:25.004994 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffad3f0-1c0c-46f2-a8cf-222e2e11d660 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:25.005500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c12f5-badb-49b1-b7b6-e9bed4713186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:25.006413 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c12f5-badb-49b1-b7b6-e9bed4713186 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:25.007136 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c12f5-badb-49b1-b7b6-e9bed4713186 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:25.007519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c12f5-badb-49b1-b7b6-e9bed4713186 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:25.020059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 274/530] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:25:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:25.020947 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffad3f0-1c0c-46f2-a8cf-222e2e11d660 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:25.028071 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c12f5-badb-49b1-b7b6-e9bed4713186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:25.029559 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c12f5-badb-49b1-b7b6-e9bed4713186 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:25.050394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-853c12f5-badb-49b1-b7b6-e9bed4713186 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:25.050394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:25.050394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:25.050394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:25.050394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:25.050394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:25.050394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:25.050394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:25.050394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:25.050394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:25.050394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:25.050394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:25.050394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:25.050394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:25.050394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:25.051589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:25.051589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:25.051589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:25.051589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:25.051589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:25.051589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:25.051589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:25.051589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:25.051589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:25.051589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:25.051589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:25.051589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:25.051589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:25.051589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:25.051589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:25.051589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:25.051589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:25.052505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:25.052505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:25.052505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:25.052505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:25.052756 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 257/531] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:25.053410 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c12f5-badb-49b1-b7b6-e9bed4713186 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:25.072318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:25.148953 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffad3f0-1c0c-46f2-a8cf-222e2e11d660 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:25.170389 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:25.186770 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ad387-b2e4-4cdd-b3b4-46b8c8679ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:25.187584 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ad387-b2e4-4cdd-b3b4-46b8c8679ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:25.187868 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ad387-b2e4-4cdd-b3b4-46b8c8679ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:25.188145 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ad387-b2e4-4cdd-b3b4-46b8c8679ee5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:25.188321 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c12f5-badb-49b1-b7b6-e9bed4713186 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:25.201800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ad387-b2e4-4cdd-b3b4-46b8c8679ee5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:25.203618 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ad387-b2e4-4cdd-b3b4-46b8c8679ee5 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:25.223047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-ad8ad387-b2e4-4cdd-b3b4-46b8c8679ee5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:25.223047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:25.223047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:25.223047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:25.223047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:25.223047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:25.223047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:25.223047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:25.223047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:25.223047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:25.223047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:25.223047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:25.223047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:25.223047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:25.223047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:25.224093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:25.224093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:25.224093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:25.224093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:25.224093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:25.224093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:25.224093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:25.224093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:25.224093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:25.224093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:25.224093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:25.224093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:25.224093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:25.224093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:25.224093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:25.224093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:25.224093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:25.225117 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:25.225117 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:25.225117 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:25.225117 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:25.225117 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 275/532] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:25.225117 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ad387-b2e4-4cdd-b3b4-46b8c8679ee5 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:25.228163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a65ddd-c4fb-4d44-9d9d-3ffab522e850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:25.228986 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a65ddd-c4fb-4d44-9d9d-3ffab522e850 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:25.229413 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a65ddd-c4fb-4d44-9d9d-3ffab522e850 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:25.229829 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a65ddd-c4fb-4d44-9d9d-3ffab522e850 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:25.231112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a65ddd-c4fb-4d44-9d9d-3ffab522e850 None admin] RBAC: Adding query filter params (role.id=5956aa478bfc4d03919395e637e61c83, scope.project.id=f965a93a2f674598890bd8cafededb2c, user.id=d8a4b0dd4e21414d91fe7bed478d15e7) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:25.231598 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a65ddd-c4fb-4d44-9d9d-3ffab522e850 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:25.233423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a65ddd-c4fb-4d44-9d9d-3ffab522e850 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:25.246890 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 258/533] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:25:25 2022] GET /identity/v3/role_assignments?user.id=d8a4b0dd4e21414d91fe7bed478d15e7&scope.project.id=f965a93a2f674598890bd8cafededb2c&role.id=5956aa478bfc4d03919395e637e61c83 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:25.248126 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a65ddd-c4fb-4d44-9d9d-3ffab522e850 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:25.341913 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ad387-b2e4-4cdd-b3b4-46b8c8679ee5 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:25.344943 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:25.390319 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986fd414-2a49-4869-8a6a-cef0df43f584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:25.391061 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fd414-2a49-4869-8a6a-cef0df43f584 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:25.391400 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fd414-2a49-4869-8a6a-cef0df43f584 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:25.391693 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fd414-2a49-4869-8a6a-cef0df43f584 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:25.392559 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986fd414-2a49-4869-8a6a-cef0df43f584 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:25.392884 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986fd414-2a49-4869-8a6a-cef0df43f584 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:25.394312 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986fd414-2a49-4869-8a6a-cef0df43f584 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:25.410252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 276/534] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:25.415298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986fd414-2a49-4869-8a6a-cef0df43f584 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:25.500570 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a65ddd-c4fb-4d44-9d9d-3ffab522e850 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:25.530086 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986fd414-2a49-4869-8a6a-cef0df43f584 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:25.541289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4363ed-98ec-4234-8b67-90fb6a1e6060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:25.542423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4363ed-98ec-4234-8b67-90fb6a1e6060 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:25.542769 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4363ed-98ec-4234-8b67-90fb6a1e6060 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:25.543075 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4363ed-98ec-4234-8b67-90fb6a1e6060 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:25.544433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4363ed-98ec-4234-8b67-90fb6a1e6060 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:25.544751 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4363ed-98ec-4234-8b67-90fb6a1e6060 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:25.546382 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4363ed-98ec-4234-8b67-90fb6a1e6060 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:25.563644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 259/535] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:25.573684 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4363ed-98ec-4234-8b67-90fb6a1e6060 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:25.581666 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3695472-37a7-4b16-a0e5-ae5374c14caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:25.582539 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3695472-37a7-4b16-a0e5-ae5374c14caa None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:25.582862 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3695472-37a7-4b16-a0e5-ae5374c14caa None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:25.583144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3695472-37a7-4b16-a0e5-ae5374c14caa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:25.596522 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3695472-37a7-4b16-a0e5-ae5374c14caa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:25.601392 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:25.601990 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3695472-37a7-4b16-a0e5-ae5374c14caa None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:25.620483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-d3695472-37a7-4b16-a0e5-ae5374c14caa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:25.620483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:25.620483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:25.620483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:25.620483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:25.620483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:25.620483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:25.620483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:25.620483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:25.620483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:25.620483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:25.620483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:25.620483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:25:25.620483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:25:25.620483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:25:25.621938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:25:25.621938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:25.621938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:25.621938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:25.621938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:25.621938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:25.621938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:25.621938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:25.621938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:25.621938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:25.621938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:25.621938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:25.621938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:25.621938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:25.621938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:25.621938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:25.621938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:25.622885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:25:25.622885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:25:25.622885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:25:25.622885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:25:25.622885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:25:25.622885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:25:25.622885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:25.622885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:25.622885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 277/536] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:25.628013 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3695472-37a7-4b16-a0e5-ae5374c14caa None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:25.643155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:25.743909 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3695472-37a7-4b16-a0e5-ae5374c14caa None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:25.785867 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3fc2e7-6083-4186-a219-87326f57f84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:25.786965 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3fc2e7-6083-4186-a219-87326f57f84a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:25.787477 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3fc2e7-6083-4186-a219-87326f57f84a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:25.788026 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3fc2e7-6083-4186-a219-87326f57f84a None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:25.789317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3fc2e7-6083-4186-a219-87326f57f84a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:25.789845 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3fc2e7-6083-4186-a219-87326f57f84a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:25.793118 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3fc2e7-6083-4186-a219-87326f57f84a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:25.807248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 278/537] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:25:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:25.812905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3fc2e7-6083-4186-a219-87326f57f84a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:25.912662 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4363ed-98ec-4234-8b67-90fb6a1e6060 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:25.925540 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3fc2e7-6083-4186-a219-87326f57f84a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:25.939080 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:25.965666 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c02c27-7153-4004-81aa-bbd36639c235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:25.967276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c02c27-7153-4004-81aa-bbd36639c235 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:25.967771 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c02c27-7153-4004-81aa-bbd36639c235 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:25.968249 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c02c27-7153-4004-81aa-bbd36639c235 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:25.969287 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c02c27-7153-4004-81aa-bbd36639c235 None admin] RBAC: Adding query filter params (role.id=5956aa478bfc4d03919395e637e61c83, scope.project.id=f965a93a2f674598890bd8cafededb2c, user.id=990999672d1a4f19896a143e1eeddd44) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:25.969805 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c02c27-7153-4004-81aa-bbd36639c235 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:25.971497 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c02c27-7153-4004-81aa-bbd36639c235 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:25.977936 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 279/538] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:25:25 2022] GET /identity/v3/role_assignments?user.id=990999672d1a4f19896a143e1eeddd44&scope.project.id=f965a93a2f674598890bd8cafededb2c&role.id=5956aa478bfc4d03919395e637e61c83 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:26.013140 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5398fc7-d9ce-46b1-9958-416938460f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:26.013953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5398fc7-d9ce-46b1-9958-416938460f13 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:26.014237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5398fc7-d9ce-46b1-9958-416938460f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:26.014515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5398fc7-d9ce-46b1-9958-416938460f13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:26.031991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5398fc7-d9ce-46b1-9958-416938460f13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:26.033461 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5398fc7-d9ce-46b1-9958-416938460f13 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:26.055055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-c5398fc7-d9ce-46b1-9958-416938460f13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:26.055055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:26.055055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:26.055055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:26.055055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:26.055055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:26.055055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:26.055055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:26.055055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:26.055055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:26.055055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:26.055055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:26.055055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:25:26.055055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:25:26.055055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:25:26.055993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:25:26.055993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:26.055993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:26.055993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:26.055993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:26.055993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:26.055993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:26.055993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:26.055993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:26.055993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:26.055993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:26.055993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:26.055993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:26.055993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:26.055993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:26.055993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:26.055993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:26.056825 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:25:26.056825 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:25:26.056825 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:25:26.056825 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:25:26.056825 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:25:26.056825 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:25:26.056825 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:26.056825 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:26.056825 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 260/539] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 483 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:26.064397 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c02c27-7153-4004-81aa-bbd36639c235 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:26.208703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c02c27-7153-4004-81aa-bbd36639c235 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:26.247553 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e064ed3-de9c-4d5e-a800-7611cd900092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:26.248597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e064ed3-de9c-4d5e-a800-7611cd900092 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:26.249085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e064ed3-de9c-4d5e-a800-7611cd900092 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:26.249583 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e064ed3-de9c-4d5e-a800-7611cd900092 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:26.250574 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e064ed3-de9c-4d5e-a800-7611cd900092 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:26.251156 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e064ed3-de9c-4d5e-a800-7611cd900092 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:26.252757 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e064ed3-de9c-4d5e-a800-7611cd900092 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:26.254696 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:26.263859 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f6ab6-0f9b-4568-81c6-9eedbcc4b853 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:26.264221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f6ab6-0f9b-4568-81c6-9eedbcc4b853 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:26.264425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f6ab6-0f9b-4568-81c6-9eedbcc4b853 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:26.266108 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 261/540] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:26.270499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 280/541] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:25:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:26.271698 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a3adc-ee08-4eab-a5dd-3cb409567a86 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:26.271988 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a3adc-ee08-4eab-a5dd-3cb409567a86 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:26.272341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a3adc-ee08-4eab-a5dd-3cb409567a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:26.273372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e064ed3-de9c-4d5e-a800-7611cd900092 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:26.293461 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ee1a3adc-ee08-4eab-a5dd-3cb409567a86 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:26.294671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ee1a3adc-ee08-4eab-a5dd-3cb409567a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:26.413701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:26.421177 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 262/542] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 31 14:25:26.434611 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e064ed3-de9c-4d5e-a800-7611cd900092 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:26.483412 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ca233-353b-42c2-9391-f345c2094b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:26.484000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ca233-353b-42c2-9391-f345c2094b31 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:26.484284 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ca233-353b-42c2-9391-f345c2094b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:26.484565 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ca233-353b-42c2-9391-f345c2094b31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:26.485400 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ca233-353b-42c2-9391-f345c2094b31 None admin] RBAC: Adding query filter params (role.id=5956aa478bfc4d03919395e637e61c83, scope.project.id=f965a93a2f674598890bd8cafededb2c, user.id=8966bc2ea7ff497c84214c9c9e6d53a8) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:26.485712 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ca233-353b-42c2-9391-f345c2094b31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:26.487162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ca233-353b-42c2-9391-f345c2094b31 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:26.493131 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 281/543] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:25:26 2022] GET /identity/v3/role_assignments?user.id=8966bc2ea7ff497c84214c9c9e6d53a8&scope.project.id=f965a93a2f674598890bd8cafededb2c&role.id=5956aa478bfc4d03919395e637e61c83 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:26.498734 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a3adc-ee08-4eab-a5dd-3cb409567a86 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:26.616356 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a3adc-ee08-4eab-a5dd-3cb409567a86 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:26.662174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf15042-2407-4709-af7f-833905e8c03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:26.663035 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf15042-2407-4709-af7f-833905e8c03a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:26.663318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf15042-2407-4709-af7f-833905e8c03a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:26.663590 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf15042-2407-4709-af7f-833905e8c03a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:26.676118 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf15042-2407-4709-af7f-833905e8c03a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:26.684008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-bbf15042-2407-4709-af7f-833905e8c03a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:26.684008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:26.684008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:26.684008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:26.684008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:26.684008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:26.684008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:26.684008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:26.684008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:26.684008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:26.684008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:26.684008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:26.684008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:26.684008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:26.684008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:26.684008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:26.685187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:26.685187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:26.685187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:26.685187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:26.685187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:26.685187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:26.685187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:26.685187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:26.685187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:26.685187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:26.685187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:26.685187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:26.685187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:26.685187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:26.685187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:26.685187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:26.686175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:26.686175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:26.686175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:26.686175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:26.686175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:26.686175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:26.686175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:26.686175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:26.686175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:26.686175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:26.686175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:26.686175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:26.686175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 263/544] 10.222.0.67 () {56 vars in 1102 bytes} [Tue May 31 14:25:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:26.691316 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ca233-353b-42c2-9391-f345c2094b31 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:26.756395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:26.802433 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ca233-353b-42c2-9391-f345c2094b31 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:26.837131 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa214f-4905-42d7-9719-3fef2d119824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:26.837807 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa214f-4905-42d7-9719-3fef2d119824 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:26.838090 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa214f-4905-42d7-9719-3fef2d119824 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:26.838366 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa214f-4905-42d7-9719-3fef2d119824 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:26.839156 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa214f-4905-42d7-9719-3fef2d119824 None admin] RBAC: Adding query filter params (name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:26.839501 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa214f-4905-42d7-9719-3fef2d119824 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:26.841458 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa214f-4905-42d7-9719-3fef2d119824 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:26.850320 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 282/545] 10.222.0.67 () {56 vars in 1103 bytes} [Tue May 31 14:25:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:26.855852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf15042-2407-4709-af7f-833905e8c03a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:26.928928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:26.992577 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf15042-2407-4709-af7f-833905e8c03a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:27.032190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb83f4f-19a7-4447-b7d3-bbce6589691e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:27.032921 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb83f4f-19a7-4447-b7d3-bbce6589691e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:27.033205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb83f4f-19a7-4447-b7d3-bbce6589691e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:27.033481 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb83f4f-19a7-4447-b7d3-bbce6589691e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:27.044236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb83f4f-19a7-4447-b7d3-bbce6589691e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:27.045681 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb83f4f-19a7-4447-b7d3-bbce6589691e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:27.063314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-feb83f4f-19a7-4447-b7d3-bbce6589691e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:27.063314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:27.063314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:27.063314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:27.063314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:27.063314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:27.063314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:27.063314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:27.063314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:27.063314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:27.063314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:27.063314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:27.063314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:27.063314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:27.063314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:27.064185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:27.064185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:27.064185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:27.064185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:27.064185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:27.064185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:27.064185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:27.064185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:27.064185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:27.064185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:27.064185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:27.064185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:27.064185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:27.064185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:27.064185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:27.064185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:27.064185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:27.065033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:27.065033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:27.065033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:27.065033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:27.065033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 264/546] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:25:27.070376 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa214f-4905-42d7-9719-3fef2d119824 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:27.172508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa214f-4905-42d7-9719-3fef2d119824 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:27.193100 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:27.208378 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e881e2-1463-4caf-af38-bac18afce7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:27.209052 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e881e2-1463-4caf-af38-bac18afce7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:27.209333 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e881e2-1463-4caf-af38-bac18afce7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:27.209648 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e881e2-1463-4caf-af38-bac18afce7c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:27.210382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e881e2-1463-4caf-af38-bac18afce7c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:27.210705 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e881e2-1463-4caf-af38-bac18afce7c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:27.212084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e881e2-1463-4caf-af38-bac18afce7c1 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:27.224823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 283/547] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:27.230791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb83f4f-19a7-4447-b7d3-bbce6589691e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:27.366994 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb83f4f-19a7-4447-b7d3-bbce6589691e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:27.376962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e227100-16c8-41e6-b6e1-d7f908612b3c None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:27.377245 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e227100-16c8-41e6-b6e1-d7f908612b3c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:27.377515 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e227100-16c8-41e6-b6e1-d7f908612b3c None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:27.379029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 284/548] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:27.386554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:27.388428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db4311-d852-4683-8ea2-59c38afe2a92 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:27.388711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db4311-d852-4683-8ea2-59c38afe2a92 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:27.388984 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db4311-d852-4683-8ea2-59c38afe2a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:27.411735 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-73db4311-d852-4683-8ea2-59c38afe2a92 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:27.412928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-73db4311-d852-4683-8ea2-59c38afe2a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:27.437731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3119f4dd-e547-4aa6-bf22-954ff46ec77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:27.438546 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119f4dd-e547-4aa6-bf22-954ff46ec77b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:27.438854 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119f4dd-e547-4aa6-bf22-954ff46ec77b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:27.439129 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119f4dd-e547-4aa6-bf22-954ff46ec77b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:27.455912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3119f4dd-e547-4aa6-bf22-954ff46ec77b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:27.457378 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3119f4dd-e547-4aa6-bf22-954ff46ec77b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:27.485872 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 285/549] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 31 14:25:27.491516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-3119f4dd-e547-4aa6-bf22-954ff46ec77b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 31 14:25:27.491516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:27.491516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:27.491516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:27.491516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:27.491516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:27.491516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:27.491516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:27.491516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:27.491516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:27.491516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:27.491516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:27.491516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:25:27.491516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:25:27.491516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:25:27.491516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:25:27.492958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:27.492958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:27.492958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:25:27.492958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:27.492958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:25:27.492958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:27.492958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:27.492958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:27.492958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:27.492958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:27.492958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:27.492958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:27.492958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:27.492958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:27.492958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:27.492958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:27.492958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:27.492958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:27.494207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:27.494207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:27.494207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:25:27.494207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:25:27.494207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:25:27.494207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:25:27.494207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:25:27.494207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:25:27.494207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 31 14:25:27.494207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:27.494207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 265/550] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:25:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:27.499016 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db4311-d852-4683-8ea2-59c38afe2a92 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:27.551731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3119f4dd-e547-4aa6-bf22-954ff46ec77b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:27.619498 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db4311-d852-4683-8ea2-59c38afe2a92 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:27.635676 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:27.655302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241137f-06c1-4b52-a07a-d6501871c4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:27.655980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241137f-06c1-4b52-a07a-d6501871c4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:27.656267 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241137f-06c1-4b52-a07a-d6501871c4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:27.656543 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241137f-06c1-4b52-a07a-d6501871c4c2 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:27.657377 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241137f-06c1-4b52-a07a-d6501871c4c2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:27.657684 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241137f-06c1-4b52-a07a-d6501871c4c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:27.659205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241137f-06c1-4b52-a07a-d6501871c4c2 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:27.677939 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 286/551] 10.222.0.67 () {56 vars in 1135 bytes} [Tue May 31 14:25:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:27.683833 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241137f-06c1-4b52-a07a-d6501871c4c2 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:27.781092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3119f4dd-e547-4aa6-bf22-954ff46ec77b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:27.788395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241137f-06c1-4b52-a07a-d6501871c4c2 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:27.827260 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce004d2-4b35-43bb-92be-e8ca958c4e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:27.827918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce004d2-4b35-43bb-92be-e8ca958c4e7e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:27.828198 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce004d2-4b35-43bb-92be-e8ca958c4e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:27.828466 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce004d2-4b35-43bb-92be-e8ca958c4e7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:27.851779 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3395516-3638-4595-b0e2-647b8e4f684a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:27.853762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3395516-3638-4595-b0e2-647b8e4f684a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:27.854050 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3395516-3638-4595-b0e2-647b8e4f684a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:27.854323 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3395516-3638-4595-b0e2-647b8e4f684a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:27.878732 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3395516-3638-4595-b0e2-647b8e4f684a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:27.897174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e3395516-3638-4595-b0e2-647b8e4f684a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:27.897174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:27.897174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:27.897174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:27.897174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:27.897174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:27.897174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:27.897174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:27.897174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:27.897174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:27.897174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:27.897174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:27.897174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:27.897174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:27.897174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:27.897174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:27.900668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:27.900668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:27.900668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:27.900668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:27.900668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:27.900668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:27.900668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:27.900668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:27.900668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:27.900668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:27.900668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:27.900668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:27.900668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:27.900668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:27.900668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:27.900668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:27.901477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:27.901477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:27.901477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:27.901477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:27.901477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:27.901477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:27.901477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:27.901477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:27.901477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:27.901477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:27.901477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:27.901477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:27.901477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 266/552] 10.222.0.67 () {56 vars in 1102 bytes} [Tue May 31 14:25:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:27.907333 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3395516-3638-4595-b0e2-647b8e4f684a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:27.911100 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:27.952028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:27.970015 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce004d2-4b35-43bb-92be-e8ca958c4e7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:27.971854 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce004d2-4b35-43bb-92be-e8ca958c4e7e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:28.090294 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3395516-3638-4595-b0e2-647b8e4f684a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:28.138799 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 287/553] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 31 14:25:28.140352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f56c7-867a-469f-853b-e08fa16e4cb9 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:28.140790 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f56c7-867a-469f-853b-e08fa16e4cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:28.141212 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f56c7-867a-469f-853b-e08fa16e4cb9 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:28.142950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c25914-a45c-436a-9545-5f5a07c59880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:28.143616 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c25914-a45c-436a-9545-5f5a07c59880 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:28.143900 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c25914-a45c-436a-9545-5f5a07c59880 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:28.144174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c25914-a45c-436a-9545-5f5a07c59880 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:28.144930 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c25914-a45c-436a-9545-5f5a07c59880 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:28.145238 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c25914-a45c-436a-9545-5f5a07c59880 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:28.146616 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c25914-a45c-436a-9545-5f5a07c59880 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:28.148527 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 288/554] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:28.150761 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f56c7-867a-469f-853b-e08fa16e4cb9 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:28.160615 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 267/555] 10.222.0.67 () {56 vars in 1103 bytes} [Tue May 31 14:25:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:28.162161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e3008-7a93-40a0-be92-8da233182f75 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:28.162599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e3008-7a93-40a0-be92-8da233182f75 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:28.163042 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e3008-7a93-40a0-be92-8da233182f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:28.180515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-713e3008-7a93-40a0-be92-8da233182f75 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:28.181924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-713e3008-7a93-40a0-be92-8da233182f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:28.245020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:28.270198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 268/556] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:28.271378 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e3008-7a93-40a0-be92-8da233182f75 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:28.290910 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f56c7-867a-469f-853b-e08fa16e4cb9 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:28.315016 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:28.346255 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31ec78-b20f-4680-af70-ca082f35c3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:28.347571 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31ec78-b20f-4680-af70-ca082f35c3ca None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:28.347860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31ec78-b20f-4680-af70-ca082f35c3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:28.348139 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31ec78-b20f-4680-af70-ca082f35c3ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:28.364350 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d31ec78-b20f-4680-af70-ca082f35c3ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:28.366888 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d31ec78-b20f-4680-af70-ca082f35c3ca None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:28.392883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-6d31ec78-b20f-4680-af70-ca082f35c3ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:28.392883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:28.392883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:28.392883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:28.392883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:28.392883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:28.392883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:28.392883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:28.392883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:28.392883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:28.392883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:28.392883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:28.392883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:28.392883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:28.392883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:28.394000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:28.394000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:28.394000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:28.394000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:28.394000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:28.394000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:28.394000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:28.394000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:28.394000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:28.394000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:28.394000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:28.394000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:28.394000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:28.394000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:28.394000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:28.394000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:28.394000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:28.394969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:28.394969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:28.394969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:28.394969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:28.394969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 289/557] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:28.395751 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31ec78-b20f-4680-af70-ca082f35c3ca None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:28.405611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e3008-7a93-40a0-be92-8da233182f75 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:28.444493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98977c69-f721-4ab4-956f-7d8b76a35737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:28.445684 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98977c69-f721-4ab4-956f-7d8b76a35737 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:28.446233 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98977c69-f721-4ab4-956f-7d8b76a35737 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:28.446759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98977c69-f721-4ab4-956f-7d8b76a35737 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:28.459350 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98977c69-f721-4ab4-956f-7d8b76a35737 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:28.461382 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98977c69-f721-4ab4-956f-7d8b76a35737 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:28.481527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-98977c69-f721-4ab4-956f-7d8b76a35737 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:28.481527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:28.481527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:28.481527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:28.481527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:28.481527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:28.481527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:28.481527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:28.481527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:28.481527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:28.481527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:28.481527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:28.481527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:28.481527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:28.481527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:28.482929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:28.482929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:28.482929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:28.482929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:28.482929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:28.482929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:28.482929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:28.482929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:28.482929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:28.482929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:28.482929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:28.482929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:28.482929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:28.482929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:28.482929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:28.482929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:28.482929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:28.483814 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:28.483814 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:28.483814 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:28.483814 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:28.491567 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 269/558] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:28.493792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98977c69-f721-4ab4-956f-7d8b76a35737 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:28.540517 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31ec78-b20f-4680-af70-ca082f35c3ca None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:28.573508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:28.588756 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:28.594310 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a008eb-c9db-4c47-934d-5d889dac8104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:28.595301 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a008eb-c9db-4c47-934d-5d889dac8104 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:28.595587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a008eb-c9db-4c47-934d-5d889dac8104 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:28.595867 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a008eb-c9db-4c47-934d-5d889dac8104 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:28.610611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98977c69-f721-4ab4-956f-7d8b76a35737 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:28.611515 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a008eb-c9db-4c47-934d-5d889dac8104 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:28.623064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-82a008eb-c9db-4c47-934d-5d889dac8104 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:28.623064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:28.623064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:28.623064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:28.623064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:28.623064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:28.623064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:28.623064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:28.623064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:28.623064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:28.623064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:28.623064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:28.623064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:28.623064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:28.623064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:28.623064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:28.624076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:28.624076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:28.624076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:28.624076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:28.624076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:28.624076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:28.624076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:28.624076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:28.624076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:28.624076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:28.624076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:28.624076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:28.624076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:28.624076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:28.624076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:28.624076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:28.625018 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:28.625018 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:28.625018 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:28.625018 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:28.625018 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:28.625018 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:28.625018 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:28.625018 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:28.625018 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:28.625018 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:28.625018 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:28.625018 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:28.626742 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 290/559] 10.222.0.67 () {56 vars in 1102 bytes} [Tue May 31 14:25:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:28.627873 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a008eb-c9db-4c47-934d-5d889dac8104 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:28.661356 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724c1b5-2df9-4d30-89f5-255600fc4e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:28.662529 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724c1b5-2df9-4d30-89f5-255600fc4e79 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:28.663122 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724c1b5-2df9-4d30-89f5-255600fc4e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:28.663640 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724c1b5-2df9-4d30-89f5-255600fc4e79 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:28.664608 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7724c1b5-2df9-4d30-89f5-255600fc4e79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:28.665149 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7724c1b5-2df9-4d30-89f5-255600fc4e79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:28.666817 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7724c1b5-2df9-4d30-89f5-255600fc4e79 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:28.690285 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 270/560] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:28.690285 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724c1b5-2df9-4d30-89f5-255600fc4e79 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:28.765230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a008eb-c9db-4c47-934d-5d889dac8104 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:28.800934 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124787cb-59bf-4e84-81c9-236cbd266f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:28.801768 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124787cb-59bf-4e84-81c9-236cbd266f6f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:28.802153 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124787cb-59bf-4e84-81c9-236cbd266f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:28.802671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124787cb-59bf-4e84-81c9-236cbd266f6f None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:28.803386 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124787cb-59bf-4e84-81c9-236cbd266f6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:28.803795 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124787cb-59bf-4e84-81c9-236cbd266f6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:28.805261 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124787cb-59bf-4e84-81c9-236cbd266f6f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:28.815180 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724c1b5-2df9-4d30-89f5-255600fc4e79 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:28.822360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 291/561] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:28.824374 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124787cb-59bf-4e84-81c9-236cbd266f6f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:28.852944 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a3287-2770-4b36-9afb-a944c8b1fca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:28.854108 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a3287-2770-4b36-9afb-a944c8b1fca3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:28.854932 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a3287-2770-4b36-9afb-a944c8b1fca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:28.855532 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a3287-2770-4b36-9afb-a944c8b1fca3 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:28.856620 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a3287-2770-4b36-9afb-a944c8b1fca3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:28.857247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a3287-2770-4b36-9afb-a944c8b1fca3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:28.859406 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a3287-2770-4b36-9afb-a944c8b1fca3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:28.862945 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:28.870578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:28.874721 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 271/562] 10.222.0.67 () {56 vars in 1103 bytes} [Tue May 31 14:25:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:28.875837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a3287-2770-4b36-9afb-a944c8b1fca3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:28.941874 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124787cb-59bf-4e84-81c9-236cbd266f6f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:28.983634 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c253054-058d-4489-ba31-b1ed284ebec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:28.984438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c253054-058d-4489-ba31-b1ed284ebec0 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:28.984717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c253054-058d-4489-ba31-b1ed284ebec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:28.984989 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c253054-058d-4489-ba31-b1ed284ebec0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:28.992442 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c253054-058d-4489-ba31-b1ed284ebec0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:28.993901 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c253054-058d-4489-ba31-b1ed284ebec0 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:29.008227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-2c253054-058d-4489-ba31-b1ed284ebec0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:29.008227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:29.008227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:29.008227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:29.008227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:29.008227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:29.008227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:29.008227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:29.008227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:29.008227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:29.008227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:29.008227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:29.008227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:25:29.008227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:25:29.008227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:25:29.009171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:25:29.009171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:29.009171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:29.009171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:29.009171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:29.009171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:29.009171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:29.009171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:29.009171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:29.009171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:29.009171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:29.009171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:29.009171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:29.009171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:29.009171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:29.009171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:29.009171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:29.010058 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:25:29.010058 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:25:29.010058 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:25:29.010058 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:25:29.010058 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:25:29.010058 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:25:29.010058 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:29.010058 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:29.010493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a3287-2770-4b36-9afb-a944c8b1fca3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:29.014581 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 292/563] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:29.015573 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c253054-058d-4489-ba31-b1ed284ebec0 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:29.046802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e9ec2-44de-48d8-99f4-35a28b185f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:29.047604 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e9ec2-44de-48d8-99f4-35a28b185f74 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:29.047891 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e9ec2-44de-48d8-99f4-35a28b185f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:29.048169 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e9ec2-44de-48d8-99f4-35a28b185f74 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:29.055523 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e9ec2-44de-48d8-99f4-35a28b185f74 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:29.056969 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e9ec2-44de-48d8-99f4-35a28b185f74 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:29.071775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-461e9ec2-44de-48d8-99f4-35a28b185f74 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 31 14:25:29.071775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:29.071775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:29.071775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:29.071775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:29.071775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:29.071775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:29.071775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:29.071775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:29.071775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:29.071775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:29.071775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:29.071775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:25:29.071775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:25:29.071775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:25:29.071775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:25:29.072748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:29.072748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:29.072748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:25:29.072748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:29.072748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:25:29.072748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:29.072748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:29.072748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:29.072748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:29.072748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:29.072748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:29.072748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:29.072748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:29.072748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:29.072748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:29.072748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:29.072748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:29.072748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:29.073688 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:29.073688 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:29.073688 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:25:29.073688 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:25:29.073688 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:25:29.073688 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:25:29.073688 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:25:29.073688 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:25:29.073688 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 31 14:25:29.073688 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:29.073688 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 272/564] 10.222.0.67 () {56 vars in 1093 bytes} [Tue May 31 14:25:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:29.076778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b45a3-3e10-4826-9b61-1a3ec70b05c3 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:29.076778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b45a3-3e10-4826-9b61-1a3ec70b05c3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:29.076778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b45a3-3e10-4826-9b61-1a3ec70b05c3 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:29.081779 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 273/565] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:29.081779 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b45a3-3e10-4826-9b61-1a3ec70b05c3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:29.130511 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:29.133759 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c253054-058d-4489-ba31-b1ed284ebec0 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:29.144251 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:29.167890 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badf367-9390-4799-8a58-c28f635e90e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:29.168678 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badf367-9390-4799-8a58-c28f635e90e7 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:29.168955 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badf367-9390-4799-8a58-c28f635e90e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:29.169224 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badf367-9390-4799-8a58-c28f635e90e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:29.179482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0badf367-9390-4799-8a58-c28f635e90e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:29.180956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0badf367-9390-4799-8a58-c28f635e90e7 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:29.186745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b45a3-3e10-4826-9b61-1a3ec70b05c3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:29.198045 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-0badf367-9390-4799-8a58-c28f635e90e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:29.198045 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:29.198045 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:29.198045 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:29.198045 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:29.198045 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:29.198045 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:29.198045 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:29.198045 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:29.198045 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:29.198045 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:29.198045 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:29.198045 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:29.198045 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:29.198045 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:29.199532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:29.199532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:29.199532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:29.199532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:29.199532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:29.199532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:29.199532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:29.199532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:29.199532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:29.199532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:29.199532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:29.199532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:29.199532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:29.199532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:29.199532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:29.199532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:29.199532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:29.200448 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:29.200448 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:29.200448 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:29.200448 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:29.200448 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 293/566] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:29.200731 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badf367-9390-4799-8a58-c28f635e90e7 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:29.221333 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287cc5b-02d2-452e-9731-eeadba001acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:29.221994 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287cc5b-02d2-452e-9731-eeadba001acc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:29.222280 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287cc5b-02d2-452e-9731-eeadba001acc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:29.222562 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287cc5b-02d2-452e-9731-eeadba001acc None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:29.223421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287cc5b-02d2-452e-9731-eeadba001acc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:29.223737 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287cc5b-02d2-452e-9731-eeadba001acc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:29.225154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287cc5b-02d2-452e-9731-eeadba001acc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:29.242711 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 274/567] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:25:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:29.244052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4609b8-edd3-422e-a947-27c760a7d516 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:29.244350 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4609b8-edd3-422e-a947-27c760a7d516 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:29.244634 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4609b8-edd3-422e-a947-27c760a7d516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:29.260726 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-fe4609b8-edd3-422e-a947-27c760a7d516 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:29.262507 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fe4609b8-edd3-422e-a947-27c760a7d516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:29.307622 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badf367-9390-4799-8a58-c28f635e90e7 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:29.328586 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 275/568] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:29.336924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326bf7a-99f4-403a-903c-cb6080b64c31 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:29.336924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326bf7a-99f4-403a-903c-cb6080b64c31 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:29.336924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326bf7a-99f4-403a-903c-cb6080b64c31 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:29.336924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 276/569] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:29.336924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326bf7a-99f4-403a-903c-cb6080b64c31 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:29.345360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7adb21-7707-44eb-a7b2-d040f4bcf5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:29.346385 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7adb21-7707-44eb-a7b2-d040f4bcf5d5 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:29.346700 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7adb21-7707-44eb-a7b2-d040f4bcf5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:29.346984 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7adb21-7707-44eb-a7b2-d040f4bcf5d5 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:29.348201 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7adb21-7707-44eb-a7b2-d040f4bcf5d5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:29.348477 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7adb21-7707-44eb-a7b2-d040f4bcf5d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:29.349972 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7adb21-7707-44eb-a7b2-d040f4bcf5d5 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:29.367018 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 294/570] 10.222.0.67 () {56 vars in 1133 bytes} [Tue May 31 14:25:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:29.367891 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7adb21-7707-44eb-a7b2-d040f4bcf5d5 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:29.379771 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:29.387752 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:29.438433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326bf7a-99f4-403a-903c-cb6080b64c31 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:29.466414 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7adb21-7707-44eb-a7b2-d040f4bcf5d5 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:29.471624 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e26d35-54bd-4de4-ba78-6ca27e344c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:29.472287 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e26d35-54bd-4de4-ba78-6ca27e344c1a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:29.472575 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e26d35-54bd-4de4-ba78-6ca27e344c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:29.472854 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e26d35-54bd-4de4-ba78-6ca27e344c1a None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:29.473577 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e26d35-54bd-4de4-ba78-6ca27e344c1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:29.473887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e26d35-54bd-4de4-ba78-6ca27e344c1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:29.475291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e26d35-54bd-4de4-ba78-6ca27e344c1a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:29.486968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 277/571] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:29.488164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fdb69-e2d0-4ee9-a313-4b0b7a276e5c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:29.488455 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fdb69-e2d0-4ee9-a313-4b0b7a276e5c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:29.488740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fdb69-e2d0-4ee9-a313-4b0b7a276e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:29.504267 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3795846c-0d1b-419e-ad8f-d23863232389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:29.504924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3795846c-0d1b-419e-ad8f-d23863232389 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:29.505075 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5b9fdb69-e2d0-4ee9-a313-4b0b7a276e5c None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:29.505210 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3795846c-0d1b-419e-ad8f-d23863232389 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:29.505491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3795846c-0d1b-419e-ad8f-d23863232389 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:29.506250 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5b9fdb69-e2d0-4ee9-a313-4b0b7a276e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:29.506355 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3795846c-0d1b-419e-ad8f-d23863232389 None admin] RBAC: Adding query filter params (role.id=5956aa478bfc4d03919395e637e61c83, scope.project.id=f965a93a2f674598890bd8cafededb2c, user.id=dfa2282f957941b98db1d7497da3351b) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:29.506635 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3795846c-0d1b-419e-ad8f-d23863232389 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:29.508084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3795846c-0d1b-419e-ad8f-d23863232389 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:29.513420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 295/572] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:25:29 2022] GET /identity/v3/role_assignments?user.id=dfa2282f957941b98db1d7497da3351b&scope.project.id=f965a93a2f674598890bd8cafededb2c&role.id=5956aa478bfc4d03919395e637e61c83 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:29.514184 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3795846c-0d1b-419e-ad8f-d23863232389 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:29.589951 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 278/573] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:29.591124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fdb69-e2d0-4ee9-a313-4b0b7a276e5c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:29.608346 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:29.639113 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3795846c-0d1b-419e-ad8f-d23863232389 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:29.642400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:29.680634 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2a4f4-e3a3-4347-ba22-7a59e97559b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:29.681458 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2a4f4-e3a3-4347-ba22-7a59e97559b1 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:29.681738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2a4f4-e3a3-4347-ba22-7a59e97559b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:29.682010 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2a4f4-e3a3-4347-ba22-7a59e97559b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:29.694670 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2a4f4-e3a3-4347-ba22-7a59e97559b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:29.704738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-8dc2a4f4-e3a3-4347-ba22-7a59e97559b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:29.704738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:29.704738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:29.704738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:29.704738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:29.704738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:29.704738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:29.704738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:29.704738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:29.704738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:29.704738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:29.704738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:29.704738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:29.704738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:29.704738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:29.704738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:29.705860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:29.705860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:29.705860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:29.705860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:29.705860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:29.705860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:29.705860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:29.705860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:29.705860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:29.705860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:29.705860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:29.705860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:29.705860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:29.705860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:29.705860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:29.705860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:29.706886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:29.706886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:29.706886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:29.706886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:29.706886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:29.706886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:29.706886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:29.706886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:29.706886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:29.706886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:29.706886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:29.706886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:29.710938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 296/574] 10.222.0.67 () {56 vars in 1102 bytes} [Tue May 31 14:25:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:29.711548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2a4f4-e3a3-4347-ba22-7a59e97559b1 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:29.715906 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fdb69-e2d0-4ee9-a313-4b0b7a276e5c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:29.757438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efb8b9-65ea-4348-9575-1951a7b587e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:29.758165 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efb8b9-65ea-4348-9575-1951a7b587e7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:29.758451 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efb8b9-65ea-4348-9575-1951a7b587e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:29.758759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efb8b9-65ea-4348-9575-1951a7b587e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:29.769227 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6efb8b9-65ea-4348-9575-1951a7b587e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:29.770786 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6efb8b9-65ea-4348-9575-1951a7b587e7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:29.789860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e6efb8b9-65ea-4348-9575-1951a7b587e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:29.789860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:29.789860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:29.789860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:29.789860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:29.789860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:29.789860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:29.789860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:29.789860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:29.789860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:29.789860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:29.789860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:29.789860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:29.789860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:29.789860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:29.790815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:29.790815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:29.790815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:29.790815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:29.790815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:29.790815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:29.790815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:29.790815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:29.790815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:29.790815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:29.790815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:29.790815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:29.790815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:29.790815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:29.790815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:29.790815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:29.790815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:29.791709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:29.791709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:29.791709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:29.791709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:29.794995 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 279/575] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:29.796666 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efb8b9-65ea-4348-9575-1951a7b587e7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:29.816503 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2a4f4-e3a3-4347-ba22-7a59e97559b1 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:29.853098 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e5bb0-a192-403f-9a77-80357a20c227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:29.853912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e5bb0-a192-403f-9a77-80357a20c227 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:29.854196 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e5bb0-a192-403f-9a77-80357a20c227 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:29.854472 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e5bb0-a192-403f-9a77-80357a20c227 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:29.857803 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:29.866108 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e5bb0-a192-403f-9a77-80357a20c227 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:29.868253 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e5bb0-a192-403f-9a77-80357a20c227 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:29.886508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-564e5bb0-a192-403f-9a77-80357a20c227 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 31 14:25:29.886508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:29.886508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:29.886508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:29.886508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:29.886508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:29.886508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:29.886508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:29.886508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:29.886508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:29.886508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:29.886508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:29.886508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:25:29.886508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:25:29.886508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:25:29.886508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:25:29.887619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:29.887619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:29.887619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:25:29.887619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:29.887619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:25:29.887619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:29.887619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:29.887619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:29.887619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:29.887619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:29.887619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:29.887619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:29.887619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:29.887619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:29.887619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:29.887619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:29.887619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:29.887619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:29.893602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:29.893602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:29.893602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:25:29.893602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:25:29.893602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:25:29.893602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:25:29.893602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:25:29.893602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:25:29.893602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 31 14:25:29.893602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:29.893602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 297/576] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:25:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:29.893602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e5bb0-a192-403f-9a77-80357a20c227 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:29.895897 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:29.900895 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efb8b9-65ea-4348-9575-1951a7b587e7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:29.939318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d548289f-f9c6-4384-94c6-3df24b0d933f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:29.940430 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548289f-f9c6-4384-94c6-3df24b0d933f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:29.940799 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548289f-f9c6-4384-94c6-3df24b0d933f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:29.941043 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548289f-f9c6-4384-94c6-3df24b0d933f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:29.952434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d548289f-f9c6-4384-94c6-3df24b0d933f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:29.960130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-d548289f-f9c6-4384-94c6-3df24b0d933f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:29.960130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:29.960130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:29.960130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:29.960130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:29.960130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:29.960130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:29.960130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:29.960130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:29.960130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:29.960130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:29.960130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:29.960130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:29.960130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:29.960130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:29.960130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:29.961063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:29.961063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:29.961063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:29.961063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:29.961063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:29.961063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:29.961063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:29.961063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:29.961063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:29.961063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:29.961063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:29.961063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:29.961063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:29.961063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:29.961063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:29.961063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:29.961882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:29.961882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:29.961882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:29.961882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:29.961882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:29.961882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:29.961882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:29.961882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:29.961882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:29.961882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:29.961882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:29.961882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:29.961882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 280/577] 10.222.0.67 () {56 vars in 1102 bytes} [Tue May 31 14:25:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:29.962572 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d548289f-f9c6-4384-94c6-3df24b0d933f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:30.006187 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e5bb0-a192-403f-9a77-80357a20c227 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:30.042363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c47b46-6ac3-483c-8b42-7ddec450e5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:30.043513 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c47b46-6ac3-483c-8b42-7ddec450e5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:30.043797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c47b46-6ac3-483c-8b42-7ddec450e5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:30.044067 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c47b46-6ac3-483c-8b42-7ddec450e5e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:30.044810 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c47b46-6ac3-483c-8b42-7ddec450e5e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:30.045112 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c47b46-6ac3-483c-8b42-7ddec450e5e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:30.046483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c47b46-6ac3-483c-8b42-7ddec450e5e8 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:30.054398 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 298/578] 10.222.0.67 () {56 vars in 1103 bytes} [Tue May 31 14:25:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:30.055211 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c47b46-6ac3-483c-8b42-7ddec450e5e8 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:30.074502 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d548289f-f9c6-4384-94c6-3df24b0d933f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:30.112394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50140d61-5a86-4c48-897b-c305f0131c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:30.113069 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50140d61-5a86-4c48-897b-c305f0131c23 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:30.113350 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50140d61-5a86-4c48-897b-c305f0131c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:30.113626 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50140d61-5a86-4c48-897b-c305f0131c23 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:30.114343 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50140d61-5a86-4c48-897b-c305f0131c23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:30.114676 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50140d61-5a86-4c48-897b-c305f0131c23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:30.116033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50140d61-5a86-4c48-897b-c305f0131c23 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:30.120788 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:30.137495 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 281/579] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:30.137495 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50140d61-5a86-4c48-897b-c305f0131c23 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:30.147250 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:30.181166 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c47b46-6ac3-483c-8b42-7ddec450e5e8 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:30.216920 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b0840-db74-4c1b-8b9d-add9fb62f075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:30.217679 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b0840-db74-4c1b-8b9d-add9fb62f075 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:30.218028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b0840-db74-4c1b-8b9d-add9fb62f075 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:30.218371 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b0840-db74-4c1b-8b9d-add9fb62f075 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:30.220927 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b0840-db74-4c1b-8b9d-add9fb62f075 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:30.221446 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b0840-db74-4c1b-8b9d-add9fb62f075 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:30.223941 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b0840-db74-4c1b-8b9d-add9fb62f075 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:30.256458 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 299/580] 10.222.0.67 () {56 vars in 1137 bytes} [Tue May 31 14:25:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:30.256458 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b0840-db74-4c1b-8b9d-add9fb62f075 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:30.260640 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50140d61-5a86-4c48-897b-c305f0131c23 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:30.314612 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12529a73-5e1e-4d03-9e3b-6aa7f40a3079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:30.315243 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12529a73-5e1e-4d03-9e3b-6aa7f40a3079 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:30.315533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12529a73-5e1e-4d03-9e3b-6aa7f40a3079 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:30.315788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12529a73-5e1e-4d03-9e3b-6aa7f40a3079 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:30.316536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12529a73-5e1e-4d03-9e3b-6aa7f40a3079 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:30.316828 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12529a73-5e1e-4d03-9e3b-6aa7f40a3079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:30.318187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12529a73-5e1e-4d03-9e3b-6aa7f40a3079 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:30.331696 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 282/581] 10.222.0.67 () {56 vars in 1103 bytes} [Tue May 31 14:25:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:30.331696 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12529a73-5e1e-4d03-9e3b-6aa7f40a3079 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:30.376602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b0840-db74-4c1b-8b9d-add9fb62f075 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:30.391616 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:30.408821 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:30.414379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b74cbbb-f12f-4eb1-8b2b-6f564fef5200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:30.415204 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74cbbb-f12f-4eb1-8b2b-6f564fef5200 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:30.415488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74cbbb-f12f-4eb1-8b2b-6f564fef5200 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:30.415770 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74cbbb-f12f-4eb1-8b2b-6f564fef5200 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:30.426081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b74cbbb-f12f-4eb1-8b2b-6f564fef5200 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:30.427618 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b74cbbb-f12f-4eb1-8b2b-6f564fef5200 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:30.440732 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12529a73-5e1e-4d03-9e3b-6aa7f40a3079 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:30.446170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-2b74cbbb-f12f-4eb1-8b2b-6f564fef5200 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:30.446170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:30.446170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:30.446170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:30.446170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:30.446170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:30.446170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:30.446170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:30.446170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:30.446170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:30.446170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:30.446170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:30.446170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:30.446170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:30.446170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:30.447123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:30.447123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:30.447123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:30.447123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:30.447123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:30.447123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:30.447123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:30.447123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:30.447123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:30.447123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:30.447123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:30.447123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:30.447123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:30.447123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:30.447123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:30.447123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:30.447123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:30.448027 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:30.448027 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:30.448027 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:30.448027 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:30.450957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 300/582] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:30.452630 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b74cbbb-f12f-4eb1-8b2b-6f564fef5200 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:30.478631 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933645db-9062-4a4b-9e9c-d4e12cc9ad87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:30.479507 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933645db-9062-4a4b-9e9c-d4e12cc9ad87 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:30.479793 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933645db-9062-4a4b-9e9c-d4e12cc9ad87 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:30.480237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933645db-9062-4a4b-9e9c-d4e12cc9ad87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:30.486952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933645db-9062-4a4b-9e9c-d4e12cc9ad87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:30.488490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933645db-9062-4a4b-9e9c-d4e12cc9ad87 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:30.502533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-933645db-9062-4a4b-9e9c-d4e12cc9ad87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:30.502533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:30.502533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:30.502533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:30.502533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:30.502533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:30.502533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:30.502533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:30.502533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:30.502533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:30.502533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:30.502533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:30.502533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:25:30.502533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:25:30.502533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:25:30.503484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:25:30.503484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:30.503484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:30.503484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:30.503484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:30.503484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:30.503484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:30.503484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:30.503484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:30.503484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:30.503484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:30.503484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:30.503484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:30.503484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:30.503484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:30.503484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:30.503484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:30.504414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:25:30.504414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:25:30.504414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:25:30.504414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:25:30.504414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:25:30.504414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:25:30.504414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:30.504414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:30.504414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 283/583] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:30.504923 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933645db-9062-4a4b-9e9c-d4e12cc9ad87 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:30.561603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b74cbbb-f12f-4eb1-8b2b-6f564fef5200 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:30.599303 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6112cd8-b0aa-42ac-98b3-214e4a4c81c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:30.600023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6112cd8-b0aa-42ac-98b3-214e4a4c81c1 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:30.600311 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6112cd8-b0aa-42ac-98b3-214e4a4c81c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:30.600594 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6112cd8-b0aa-42ac-98b3-214e4a4c81c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:30.609581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933645db-9062-4a4b-9e9c-d4e12cc9ad87 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:30.611174 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6112cd8-b0aa-42ac-98b3-214e4a4c81c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:30.612664 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6112cd8-b0aa-42ac-98b3-214e4a4c81c1 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:30.632599 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-a6112cd8-b0aa-42ac-98b3-214e4a4c81c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:30.632599 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:30.632599 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:30.632599 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:30.632599 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:30.632599 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:30.632599 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:30.632599 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:30.632599 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:30.632599 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:30.632599 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:30.632599 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:30.632599 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:30.632599 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:30.632599 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:30.633566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:30.633566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:30.633566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:30.633566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:30.633566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:30.633566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:30.633566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:30.633566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:30.633566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:30.633566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:30.633566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:30.633566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:30.633566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:30.633566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:30.633566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:30.633566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:30.633566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:30.634450 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:30.634450 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:30.634450 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:30.634450 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:30.634450 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 301/584] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:30.634770 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6112cd8-b0aa-42ac-98b3-214e4a4c81c1 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:30.651224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:30.657359 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:30.659132 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c83d5-a92b-4750-a0d6-b0b5dfd7692d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:30.660158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c83d5-a92b-4750-a0d6-b0b5dfd7692d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:30.660615 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c83d5-a92b-4750-a0d6-b0b5dfd7692d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:30.661428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c83d5-a92b-4750-a0d6-b0b5dfd7692d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:30.674491 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c83d5-a92b-4750-a0d6-b0b5dfd7692d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:30.675994 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c83d5-a92b-4750-a0d6-b0b5dfd7692d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:30.695135 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-eb5c83d5-a92b-4750-a0d6-b0b5dfd7692d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:30.695135 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:30.695135 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:30.695135 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:30.695135 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:30.695135 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:30.695135 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:30.695135 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:30.695135 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:30.695135 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:30.695135 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:30.695135 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:30.695135 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:30.695135 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:30.695135 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:30.696067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:30.696067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:30.696067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:30.696067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:30.696067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:30.696067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:30.696067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:30.696067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:30.696067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:30.696067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:30.696067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:30.696067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:30.696067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:30.696067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:30.696067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:30.696067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:30.696067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:30.696929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:30.696929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:30.696929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:30.696929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:30.696929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 284/585] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:30.697209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c83d5-a92b-4750-a0d6-b0b5dfd7692d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:30.750599 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6112cd8-b0aa-42ac-98b3-214e4a4c81c1 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:30.785707 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a0035-2b3a-4264-8644-3f1559765f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:30.786373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a0035-2b3a-4264-8644-3f1559765f5e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:30.786671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a0035-2b3a-4264-8644-3f1559765f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:30.786960 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a0035-2b3a-4264-8644-3f1559765f5e None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:30.787690 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a0035-2b3a-4264-8644-3f1559765f5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:30.788004 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a0035-2b3a-4264-8644-3f1559765f5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:30.789387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a0035-2b3a-4264-8644-3f1559765f5e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:30.801087 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 302/586] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:30.801822 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a0035-2b3a-4264-8644-3f1559765f5e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:30.811349 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c83d5-a92b-4750-a0d6-b0b5dfd7692d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:30.847755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d53207-d8d5-4235-88f7-e8dd2f85d06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:30.848433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d53207-d8d5-4235-88f7-e8dd2f85d06c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:30.848719 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d53207-d8d5-4235-88f7-e8dd2f85d06c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:30.848994 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d53207-d8d5-4235-88f7-e8dd2f85d06c None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:30.849837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d53207-d8d5-4235-88f7-e8dd2f85d06c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:30.850113 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d53207-d8d5-4235-88f7-e8dd2f85d06c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:30.851570 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d53207-d8d5-4235-88f7-e8dd2f85d06c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:30.868276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 285/587] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:25:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:30.868276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d53207-d8d5-4235-88f7-e8dd2f85d06c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:30.894666 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:30.914569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:30.920102 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a0035-2b3a-4264-8644-3f1559765f5e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:30.955301 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3097fa2-9d85-49ab-98bd-111143ddbdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:30.955958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3097fa2-9d85-49ab-98bd-111143ddbdef None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:30.956240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3097fa2-9d85-49ab-98bd-111143ddbdef None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:30.956511 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3097fa2-9d85-49ab-98bd-111143ddbdef None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:30.957213 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3097fa2-9d85-49ab-98bd-111143ddbdef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:30.957521 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3097fa2-9d85-49ab-98bd-111143ddbdef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:30.958992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3097fa2-9d85-49ab-98bd-111143ddbdef None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:30.978131 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 303/588] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:30.978131 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3097fa2-9d85-49ab-98bd-111143ddbdef None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:30.983714 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d53207-d8d5-4235-88f7-e8dd2f85d06c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:31.018372 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4ad47-9c61-454b-b39d-f5e6dfa77b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:31.019076 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4ad47-9c61-454b-b39d-f5e6dfa77b4a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:31.019366 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4ad47-9c61-454b-b39d-f5e6dfa77b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:31.019646 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4ad47-9c61-454b-b39d-f5e6dfa77b4a None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:31.020376 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4ad47-9c61-454b-b39d-f5e6dfa77b4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:31.020689 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4ad47-9c61-454b-b39d-f5e6dfa77b4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:31.022088 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4ad47-9c61-454b-b39d-f5e6dfa77b4a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:31.034924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 286/589] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:31.035991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4ad47-9c61-454b-b39d-f5e6dfa77b4a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:31.086469 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3097fa2-9d85-49ab-98bd-111143ddbdef None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:31.129566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2cf6f-c7c4-4c67-9f3d-318c55025ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:31.130436 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2cf6f-c7c4-4c67-9f3d-318c55025ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:31.130752 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2cf6f-c7c4-4c67-9f3d-318c55025ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:31.131032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2cf6f-c7c4-4c67-9f3d-318c55025ec8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:31.138591 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2cf6f-c7c4-4c67-9f3d-318c55025ec8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:31.140068 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2cf6f-c7c4-4c67-9f3d-318c55025ec8 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:31.141919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:31.145284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4ad47-9c61-454b-b39d-f5e6dfa77b4a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:31.148673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:31.157047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-e2b2cf6f-c7c4-4c67-9f3d-318c55025ec8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 31 14:25:31.157047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:31.157047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:31.157047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:31.157047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:31.157047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:31.157047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:31.157047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:31.157047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:31.157047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:31.157047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:31.157047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:31.157047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:25:31.157047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:25:31.157047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:25:31.157047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:25:31.158170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:31.158170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:31.158170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:25:31.158170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:31.158170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:25:31.158170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:31.158170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:31.158170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:31.158170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:31.158170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:31.158170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:31.158170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:31.158170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:31.158170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:31.158170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:31.158170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:31.158170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:31.158170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:31.159159 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:31.159159 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:31.159159 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:25:31.159159 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:25:31.159159 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:25:31.159159 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:25:31.159159 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:25:31.159159 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:25:31.159159 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 31 14:25:31.159159 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:31.159159 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 304/590] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:25:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:31.159159 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2cf6f-c7c4-4c67-9f3d-318c55025ec8 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:31.189350 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0bd50b-37cd-4161-8674-b5448d543f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:31.190250 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0bd50b-37cd-4161-8674-b5448d543f56 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:31.190534 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0bd50b-37cd-4161-8674-b5448d543f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:31.190847 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0bd50b-37cd-4161-8674-b5448d543f56 None admin] PATH_INFO: `/v3/projects/f965a93a2f674598890bd8cafededb2c/users/68c6363b60954c3ebf50d2019f5f2fd2/roles/5956aa478bfc4d03919395e637e61c83` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:31.206082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0bd50b-37cd-4161-8674-b5448d543f56 None admin] RBAC: Authorizing `identity:create_grant(project_id=f965a93a2f674598890bd8cafededb2c, user_id=68c6363b60954c3ebf50d2019f5f2fd2, role_id=5956aa478bfc4d03919395e637e61c83)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:31.207550 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0bd50b-37cd-4161-8674-b5448d543f56 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:31.222354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 287/591] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:25:31 2022] PUT /identity/v3/projects/f965a93a2f674598890bd8cafededb2c/users/68c6363b60954c3ebf50d2019f5f2fd2/roles/5956aa478bfc4d03919395e637e61c83 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:25:31.223174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0bd50b-37cd-4161-8674-b5448d543f56 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:31.276858 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2cf6f-c7c4-4c67-9f3d-318c55025ec8 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:31.317805 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea02dac-bc50-4380-8371-044789840cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:31.319067 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea02dac-bc50-4380-8371-044789840cba None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:31.319600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea02dac-bc50-4380-8371-044789840cba None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:31.320084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea02dac-bc50-4380-8371-044789840cba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:31.331115 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea02dac-bc50-4380-8371-044789840cba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:31.333566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea02dac-bc50-4380-8371-044789840cba None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:31.351490 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-5ea02dac-bc50-4380-8371-044789840cba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:31.351490 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:31.351490 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:31.351490 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:31.351490 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:31.351490 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:31.351490 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:31.351490 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:31.351490 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:31.351490 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:31.351490 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:31.351490 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:31.351490 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:25:31.351490 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:25:31.351490 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:25:31.352485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:25:31.352485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:31.352485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:31.352485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:31.352485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:31.352485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:31.352485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:31.352485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:31.352485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:31.352485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:31.352485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:31.352485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:31.352485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:31.352485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:31.352485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:31.352485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:31.352485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:31.353368 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:25:31.353368 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:25:31.353368 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:25:31.353368 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:25:31.353368 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:25:31.353368 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:25:31.353368 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:31.353368 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:31.356678 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 305/592] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:31.357753 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea02dac-bc50-4380-8371-044789840cba None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:31.372785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0bd50b-37cd-4161-8674-b5448d543f56 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:31.411618 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf959e4-b7e3-4aae-8d36-c659b5313ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:31.412215 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf959e4-b7e3-4aae-8d36-c659b5313ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:31.412502 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf959e4-b7e3-4aae-8d36-c659b5313ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:31.412781 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf959e4-b7e3-4aae-8d36-c659b5313ea1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:31.421984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf959e4-b7e3-4aae-8d36-c659b5313ea1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:31.423681 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf959e4-b7e3-4aae-8d36-c659b5313ea1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:31.428011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:31.440449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-1bf959e4-b7e3-4aae-8d36-c659b5313ea1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 31 14:25:31.440449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:31.440449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:31.440449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:31.440449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:31.440449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:31.440449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:31.440449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:31.440449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:31.440449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:31.440449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:31.440449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:31.440449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:25:31.440449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:25:31.440449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:25:31.440449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:25:31.441497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:31.441497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:31.441497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:25:31.441497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:31.441497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:25:31.441497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:31.441497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:31.441497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:31.441497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:31.441497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:31.441497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:31.441497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:31.441497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:31.441497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:31.441497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:31.441497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:31.441497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:31.441497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:31.442438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:31.442438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:31.442438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:25:31.442438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:25:31.442438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:25:31.442438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:25:31.442438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:25:31.442438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:25:31.442438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 31 14:25:31.442438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:31.443028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 288/593] 10.222.0.67 () {56 vars in 1102 bytes} [Tue May 31 14:25:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:31.443788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf959e4-b7e3-4aae-8d36-c659b5313ea1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:31.462904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:31.485092 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea02dac-bc50-4380-8371-044789840cba None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:31.519921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9cf33b-a745-46f3-88c9-19d095c33b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:31.520970 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cf33b-a745-46f3-88c9-19d095c33b91 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:31.521318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cf33b-a745-46f3-88c9-19d095c33b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:31.521663 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cf33b-a745-46f3-88c9-19d095c33b91 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:31.523503 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9cf33b-a745-46f3-88c9-19d095c33b91 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:31.523895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9cf33b-a745-46f3-88c9-19d095c33b91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:31.525389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9cf33b-a745-46f3-88c9-19d095c33b91 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:31.548289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 306/594] 10.222.0.67 () {56 vars in 1137 bytes} [Tue May 31 14:25:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:31.549275 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9cf33b-a745-46f3-88c9-19d095c33b91 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:31.561279 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf959e4-b7e3-4aae-8d36-c659b5313ea1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:31.599477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71aec46-7779-45b7-ac0d-dc89a5c52495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:31.600066 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71aec46-7779-45b7-ac0d-dc89a5c52495 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:31.600352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71aec46-7779-45b7-ac0d-dc89a5c52495 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:31.600631 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71aec46-7779-45b7-ac0d-dc89a5c52495 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:31.601908 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71aec46-7779-45b7-ac0d-dc89a5c52495 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:31.601908 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71aec46-7779-45b7-ac0d-dc89a5c52495 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:31.603187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71aec46-7779-45b7-ac0d-dc89a5c52495 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:31.616522 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 289/595] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:25:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:31.617233 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71aec46-7779-45b7-ac0d-dc89a5c52495 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:31.679505 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9cf33b-a745-46f3-88c9-19d095c33b91 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:31.697669 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:31.710007 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:31.725833 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086ba85-9066-4f19-8956-93193c35d340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:31.727101 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086ba85-9066-4f19-8956-93193c35d340 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:31.727835 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086ba85-9066-4f19-8956-93193c35d340 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:31.728709 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086ba85-9066-4f19-8956-93193c35d340 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:31.730804 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5086ba85-9066-4f19-8956-93193c35d340 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:31.731577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5086ba85-9066-4f19-8956-93193c35d340 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:31.733943 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5086ba85-9066-4f19-8956-93193c35d340 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:31.738359 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71aec46-7779-45b7-ac0d-dc89a5c52495 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:31.757615 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 307/596] 10.222.0.67 () {56 vars in 1139 bytes} [Tue May 31 14:25:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:31.758750 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086ba85-9066-4f19-8956-93193c35d340 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:31.775698 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625e346-b2c4-4b8e-b9d4-56cbf54f1ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:31.776509 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625e346-b2c4-4b8e-b9d4-56cbf54f1ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:31.776830 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625e346-b2c4-4b8e-b9d4-56cbf54f1ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:31.777632 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625e346-b2c4-4b8e-b9d4-56cbf54f1ed5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:31.790746 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1625e346-b2c4-4b8e-b9d4-56cbf54f1ed5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:31.792222 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1625e346-b2c4-4b8e-b9d4-56cbf54f1ed5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:31.809367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-1625e346-b2c4-4b8e-b9d4-56cbf54f1ed5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:31.809367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:31.809367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:31.809367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:31.809367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:31.809367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:31.809367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:31.809367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:31.809367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:31.809367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:31.809367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:31.809367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:31.809367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:31.809367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:31.809367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:31.810334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:31.810334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:31.810334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:31.810334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:31.810334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:31.810334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:31.810334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:31.810334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:31.810334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:31.810334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:31.810334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:31.810334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:31.810334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:31.810334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:31.810334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:31.810334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:31.810334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:31.811286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:31.811286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:31.811286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:31.811286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:31.811286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 290/597] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:31.814956 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625e346-b2c4-4b8e-b9d4-56cbf54f1ed5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:31.869070 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086ba85-9066-4f19-8956-93193c35d340 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:31.907317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec30e56-a690-4efe-a349-ae64bb25b7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:31.908238 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec30e56-a690-4efe-a349-ae64bb25b7db None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:31.908518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec30e56-a690-4efe-a349-ae64bb25b7db None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:31.908791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec30e56-a690-4efe-a349-ae64bb25b7db None admin] PATH_INFO: `/v3/projects/f965a93a2f674598890bd8cafededb2c/users/d8a4b0dd4e21414d91fe7bed478d15e7/roles/5956aa478bfc4d03919395e637e61c83` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:31.923600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec30e56-a690-4efe-a349-ae64bb25b7db None admin] RBAC: Authorizing `identity:create_grant(project_id=f965a93a2f674598890bd8cafededb2c, user_id=d8a4b0dd4e21414d91fe7bed478d15e7, role_id=5956aa478bfc4d03919395e637e61c83)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:31.924260 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625e346-b2c4-4b8e-b9d4-56cbf54f1ed5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:31.925047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec30e56-a690-4efe-a349-ae64bb25b7db None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:31.942786 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 308/598] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:25:31 2022] PUT /identity/v3/projects/f965a93a2f674598890bd8cafededb2c/users/d8a4b0dd4e21414d91fe7bed478d15e7/roles/5956aa478bfc4d03919395e637e61c83 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:25:31.942786 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acbcd8-ca31-47d9-a5c2-557677ee8166 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:31.942786 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acbcd8-ca31-47d9-a5c2-557677ee8166 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:31.942786 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acbcd8-ca31-47d9-a5c2-557677ee8166 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:31.942786 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 309/599] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:31.942786 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31acbcd8-ca31-47d9-a5c2-557677ee8166 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:31.963347 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:31.973977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:31.977222 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f54c1-76d7-4ab7-9b4c-9afd7aa878de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:31.978337 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f54c1-76d7-4ab7-9b4c-9afd7aa878de None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:31.978844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f54c1-76d7-4ab7-9b4c-9afd7aa878de None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:31.979308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f54c1-76d7-4ab7-9b4c-9afd7aa878de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:31.994411 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f54c1-76d7-4ab7-9b4c-9afd7aa878de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:31.996351 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f54c1-76d7-4ab7-9b4c-9afd7aa878de None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:32.020625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-377f54c1-76d7-4ab7-9b4c-9afd7aa878de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:32.020625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:32.020625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:32.020625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:32.020625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:32.020625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:32.020625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:32.020625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:32.020625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:32.020625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:32.020625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:32.020625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:32.020625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:32.020625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:32.020625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:32.021643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:32.021643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:32.021643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:32.021643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:32.021643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:32.021643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:32.021643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:32.021643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:32.021643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:32.021643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:32.021643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:32.021643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:32.021643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:32.021643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:32.021643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:32.021643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:32.021643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:32.022571 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:32.022571 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:32.022571 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:32.022571 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:32.025028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 291/600] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:32.026683 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c880ed7-b779-4dad-9fda-20180b223f67 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:32.027883 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c880ed7-b779-4dad-9fda-20180b223f67 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:32.028433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c880ed7-b779-4dad-9fda-20180b223f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:32.048522 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5c880ed7-b779-4dad-9fda-20180b223f67 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:32.050250 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5c880ed7-b779-4dad-9fda-20180b223f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:32.064166 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31acbcd8-ca31-47d9-a5c2-557677ee8166 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:32.103583 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487de573-9d3b-4184-b2b8-8135e88ec5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:32.104490 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487de573-9d3b-4184-b2b8-8135e88ec5ef None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:32.104633 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487de573-9d3b-4184-b2b8-8135e88ec5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:32.104786 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487de573-9d3b-4184-b2b8-8135e88ec5ef None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:32.105566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487de573-9d3b-4184-b2b8-8135e88ec5ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:32.105871 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487de573-9d3b-4184-b2b8-8135e88ec5ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:32.107330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487de573-9d3b-4184-b2b8-8135e88ec5ef None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:32.123972 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 310/601] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:32.130032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487de573-9d3b-4184-b2b8-8135e88ec5ef None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:32.147020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 292/602] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:32.147841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c880ed7-b779-4dad-9fda-20180b223f67 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:32.223376 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:32.251287 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487de573-9d3b-4184-b2b8-8135e88ec5ef None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:32.266111 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c880ed7-b779-4dad-9fda-20180b223f67 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:32.277435 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:32.299392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000c887-d114-40b3-9181-61c78d64add8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:32.300063 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000c887-d114-40b3-9181-61c78d64add8 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:32.300351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000c887-d114-40b3-9181-61c78d64add8 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:32.300631 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000c887-d114-40b3-9181-61c78d64add8 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:32.301630 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000c887-d114-40b3-9181-61c78d64add8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:32.301963 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000c887-d114-40b3-9181-61c78d64add8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:32.304245 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000c887-d114-40b3-9181-61c78d64add8 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:32.307507 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30392dae-5a76-4f1e-ba5b-345ae83ac8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:32.308593 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30392dae-5a76-4f1e-ba5b-345ae83ac8a1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:32.309074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30392dae-5a76-4f1e-ba5b-345ae83ac8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:32.309553 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30392dae-5a76-4f1e-ba5b-345ae83ac8a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:32.317163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30392dae-5a76-4f1e-ba5b-345ae83ac8a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:32.318371 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 311/603] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:32.319114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30392dae-5a76-4f1e-ba5b-345ae83ac8a1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:32.319291 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000c887-d114-40b3-9181-61c78d64add8 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:32.336499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-30392dae-5a76-4f1e-ba5b-345ae83ac8a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:32.336499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:32.336499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:32.336499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:32.336499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:32.336499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:32.336499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:32.336499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:32.336499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:32.336499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:32.336499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:32.336499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:32.336499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:25:32.336499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:25:32.336499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:25:32.337426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:25:32.337426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:32.337426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:32.337426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:32.337426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:32.337426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:32.337426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:32.337426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:32.337426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:32.337426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:32.337426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:32.337426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:32.337426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:32.337426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:32.337426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:32.337426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:32.337426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:32.338310 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:25:32.338310 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:25:32.338310 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:25:32.338310 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:25:32.338310 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:25:32.338310 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:25:32.338310 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:32.338310 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:32.341370 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 293/604] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:32.342540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30392dae-5a76-4f1e-ba5b-345ae83ac8a1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:32.429740 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000c887-d114-40b3-9181-61c78d64add8 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:32.469366 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b170aee-4cb8-40e9-a098-95559aa0209f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:32.470216 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b170aee-4cb8-40e9-a098-95559aa0209f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:32.470577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b170aee-4cb8-40e9-a098-95559aa0209f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:32.470799 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b170aee-4cb8-40e9-a098-95559aa0209f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:32.482466 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b170aee-4cb8-40e9-a098-95559aa0209f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:32.490600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-3b170aee-4cb8-40e9-a098-95559aa0209f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:32.490600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:32.490600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:32.490600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:32.490600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:32.490600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:32.490600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:32.490600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:32.490600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:32.490600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:32.490600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:32.490600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:32.490600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:32.490600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:32.490600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:32.490600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:32.491560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:32.491560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:32.491560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:32.491560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:32.491560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:32.491560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:32.491560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:32.491560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:32.491560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:32.491560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:32.491560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:32.491560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:32.491560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:32.491560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:32.491560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:32.491560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:32.492415 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:32.492415 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:32.492415 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:32.492415 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:32.492415 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:32.492415 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:32.492415 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:32.492415 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:32.492415 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:32.492415 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:32.492415 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:32.492415 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:32.495261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30392dae-5a76-4f1e-ba5b-345ae83ac8a1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:32.497592 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 312/605] 10.222.0.67 () {56 vars in 1102 bytes} [Tue May 31 14:25:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:32.499043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b170aee-4cb8-40e9-a098-95559aa0209f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:32.503473 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:32.546334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b345b-3a2c-4e68-9fa4-546b51d9e8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:32.547209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b345b-3a2c-4e68-9fa4-546b51d9e8ce None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:32.547496 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b345b-3a2c-4e68-9fa4-546b51d9e8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:32.547772 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b345b-3a2c-4e68-9fa4-546b51d9e8ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:32.557778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883b345b-3a2c-4e68-9fa4-546b51d9e8ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:32.559856 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883b345b-3a2c-4e68-9fa4-546b51d9e8ce None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:32.580503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-883b345b-3a2c-4e68-9fa4-546b51d9e8ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:32.580503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:32.580503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:32.580503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:32.580503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:32.580503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:32.580503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:32.580503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:32.580503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:32.580503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:32.580503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:32.580503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:32.580503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:25:32.580503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:25:32.580503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:25:32.581441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:25:32.581441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:32.581441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:32.581441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:32.581441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:32.581441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:32.581441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:32.581441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:32.581441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:32.581441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:32.581441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:32.581441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:32.581441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:32.581441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:32.581441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:32.581441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:32.581441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:32.582324 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:25:32.582324 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:25:32.582324 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:25:32.582324 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:25:32.582324 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:25:32.582324 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:25:32.582324 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:32.582324 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:32.582324 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 294/606] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:32.583359 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b345b-3a2c-4e68-9fa4-546b51d9e8ce None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:32.614322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:32.622266 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b170aee-4cb8-40e9-a098-95559aa0209f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:32.666443 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdc393-b147-427f-af35-786bb1643715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:32.667136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdc393-b147-427f-af35-786bb1643715 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:32.667448 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdc393-b147-427f-af35-786bb1643715 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:32.667726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdc393-b147-427f-af35-786bb1643715 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:32.669038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdc393-b147-427f-af35-786bb1643715 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:32.669354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdc393-b147-427f-af35-786bb1643715 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:32.670840 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdc393-b147-427f-af35-786bb1643715 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:32.688185 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 313/607] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:25:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:32.689320 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdc393-b147-427f-af35-786bb1643715 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:32.696738 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b345b-3a2c-4e68-9fa4-546b51d9e8ce None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:32.737691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2d020-505f-4c0b-9965-b41bf445fa44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:32.738719 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2d020-505f-4c0b-9965-b41bf445fa44 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:32.739230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2d020-505f-4c0b-9965-b41bf445fa44 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:32.739757 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2d020-505f-4c0b-9965-b41bf445fa44 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:32.740764 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a2d020-505f-4c0b-9965-b41bf445fa44 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:32.741291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a2d020-505f-4c0b-9965-b41bf445fa44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:32.742959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a2d020-505f-4c0b-9965-b41bf445fa44 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:32.752303 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 295/608] 10.222.0.67 () {56 vars in 1103 bytes} [Tue May 31 14:25:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:32.753367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2d020-505f-4c0b-9965-b41bf445fa44 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:32.777936 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:32.825518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdc393-b147-427f-af35-786bb1643715 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:32.862179 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa42184-2f8c-4191-81a5-0fa1ac2139ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:32.863262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa42184-2f8c-4191-81a5-0fa1ac2139ed None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:32.863770 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa42184-2f8c-4191-81a5-0fa1ac2139ed None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:32.864251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa42184-2f8c-4191-81a5-0fa1ac2139ed None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:32.865256 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa42184-2f8c-4191-81a5-0fa1ac2139ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:32.865768 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa42184-2f8c-4191-81a5-0fa1ac2139ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:32.867428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa42184-2f8c-4191-81a5-0fa1ac2139ed None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:32.880247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:32.881088 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 314/609] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:25:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:32.882083 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa42184-2f8c-4191-81a5-0fa1ac2139ed None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:32.892726 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2d020-505f-4c0b-9965-b41bf445fa44 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:32.928250 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958bbe1-a9c7-471d-9e73-07de2a62981c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:32.929916 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958bbe1-a9c7-471d-9e73-07de2a62981c None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:32.930415 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958bbe1-a9c7-471d-9e73-07de2a62981c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:32.930927 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958bbe1-a9c7-471d-9e73-07de2a62981c None admin] PATH_INFO: `/v3/projects/f965a93a2f674598890bd8cafededb2c/users/990999672d1a4f19896a143e1eeddd44/roles/5956aa478bfc4d03919395e637e61c83` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:32.947087 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958bbe1-a9c7-471d-9e73-07de2a62981c None admin] RBAC: Authorizing `identity:create_grant(project_id=f965a93a2f674598890bd8cafededb2c, user_id=990999672d1a4f19896a143e1eeddd44, role_id=5956aa478bfc4d03919395e637e61c83)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:32.949525 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958bbe1-a9c7-471d-9e73-07de2a62981c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:32.966844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 296/610] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:25:32 2022] PUT /identity/v3/projects/f965a93a2f674598890bd8cafededb2c/users/990999672d1a4f19896a143e1eeddd44/roles/5956aa478bfc4d03919395e637e61c83 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:25:32.968038 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958bbe1-a9c7-471d-9e73-07de2a62981c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:33.017322 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa42184-2f8c-4191-81a5-0fa1ac2139ed None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:33.045528 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:33.059443 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd9ea6-2300-4500-b4c9-541ea77009bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:33.060597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd9ea6-2300-4500-b4c9-541ea77009bc None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:33.061111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd9ea6-2300-4500-b4c9-541ea77009bc None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:33.061598 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd9ea6-2300-4500-b4c9-541ea77009bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:33.073714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd9ea6-2300-4500-b4c9-541ea77009bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:33.075593 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd9ea6-2300-4500-b4c9-541ea77009bc None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:33.095200 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958bbe1-a9c7-471d-9e73-07de2a62981c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:33.095697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-03dd9ea6-2300-4500-b4c9-541ea77009bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:33.095697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:33.095697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:33.095697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:33.095697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:33.095697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:33.095697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:33.095697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:33.095697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:33.095697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:33.095697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:33.095697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:33.095697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:33.095697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:33.095697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:33.096587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:33.096587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:33.096587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:33.096587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:33.096587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:33.096587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:33.096587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:33.096587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:33.096587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:33.096587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:33.096587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:33.096587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:33.096587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:33.096587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:33.096587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:33.096587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:33.096587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:33.097498 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:33.097498 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:33.097498 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:33.097498 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:33.099948 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 315/611] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:33.107214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd9ea6-2300-4500-b4c9-541ea77009bc None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:33.165140 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b61fd0-3097-4a2c-beb4-a2fb2074a6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:33.166052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b61fd0-3097-4a2c-beb4-a2fb2074a6d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:33.166334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b61fd0-3097-4a2c-beb4-a2fb2074a6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:33.166610 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b61fd0-3097-4a2c-beb4-a2fb2074a6d5 None admin] PATH_INFO: `/v3/projects/f965a93a2f674598890bd8cafededb2c/users/8966bc2ea7ff497c84214c9c9e6d53a8/roles/5956aa478bfc4d03919395e637e61c83` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:33.180369 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:33.186395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b61fd0-3097-4a2c-beb4-a2fb2074a6d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=f965a93a2f674598890bd8cafededb2c, user_id=8966bc2ea7ff497c84214c9c9e6d53a8, role_id=5956aa478bfc4d03919395e637e61c83)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:33.188050 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b61fd0-3097-4a2c-beb4-a2fb2074a6d5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:33.222851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 297/612] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:25:32 2022] PUT /identity/v3/projects/f965a93a2f674598890bd8cafededb2c/users/8966bc2ea7ff497c84214c9c9e6d53a8/roles/5956aa478bfc4d03919395e637e61c83 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:25:33.225884 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd9ea6-2300-4500-b4c9-541ea77009bc None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:33.268411 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaaac0b-88a7-499b-bd12-81cde067020a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:33.269470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaaac0b-88a7-499b-bd12-81cde067020a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:33.269986 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaaac0b-88a7-499b-bd12-81cde067020a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:33.270474 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaaac0b-88a7-499b-bd12-81cde067020a None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:33.271507 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaaac0b-88a7-499b-bd12-81cde067020a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:33.272029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaaac0b-88a7-499b-bd12-81cde067020a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:33.273649 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaaac0b-88a7-499b-bd12-81cde067020a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:33.288652 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 316/613] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:33.294081 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b61fd0-3097-4a2c-beb4-a2fb2074a6d5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:33.430671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b61fd0-3097-4a2c-beb4-a2fb2074a6d5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:33.469889 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ddcb8-51fc-4edb-9b33-189e8867abed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:33.470720 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ddcb8-51fc-4edb-9b33-189e8867abed None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:33.471007 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ddcb8-51fc-4edb-9b33-189e8867abed None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:33.471285 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ddcb8-51fc-4edb-9b33-189e8867abed None admin] PATH_INFO: `/v3/users/swift` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:33.478837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ddcb8-51fc-4edb-9b33-189e8867abed None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:33.480317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ddcb8-51fc-4edb-9b33-189e8867abed None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:33.495185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-8a5ddcb8-51fc-4edb-9b33-189e8867abed None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 31 14:25:33.495185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:33.495185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:33.495185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:33.495185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:33.495185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:33.495185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:33.495185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:33.495185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:33.495185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:33.495185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:33.495185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:33.495185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:25:33.495185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:25:33.495185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:25:33.495185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:25:33.496190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:33.496190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:33.496190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:25:33.496190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:33.496190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:25:33.496190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:33.496190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:33.496190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:33.496190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:33.496190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:33.496190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:33.496190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:33.496190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:33.496190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:33.496190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:33.496190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:33.496190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:33.496190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:33.497154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:33.497154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:33.497154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:25:33.497154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:25:33.497154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:25:33.497154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:25:33.497154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:25:33.497154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:25:33.497154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 31 14:25:33.497154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:33.497154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 298/614] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:25:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:33.502426 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaaac0b-88a7-499b-bd12-81cde067020a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:33.539620 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:33.611092 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaaac0b-88a7-499b-bd12-81cde067020a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:33.658275 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe57d26-ddfd-4f2d-be74-8e5ac7b1c35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:33.658984 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe57d26-ddfd-4f2d-be74-8e5ac7b1c35b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:33.659273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe57d26-ddfd-4f2d-be74-8e5ac7b1c35b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:33.659552 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe57d26-ddfd-4f2d-be74-8e5ac7b1c35b None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:33.660406 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe57d26-ddfd-4f2d-be74-8e5ac7b1c35b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:33.660719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe57d26-ddfd-4f2d-be74-8e5ac7b1c35b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:33.662181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe57d26-ddfd-4f2d-be74-8e5ac7b1c35b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:33.689703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 317/615] 10.222.0.67 () {56 vars in 1135 bytes} [Tue May 31 14:25:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:33.690608 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ddcb8-51fc-4edb-9b33-189e8867abed None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:33.773198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:33.819483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8d7a1-b32e-4e46-9db6-9a5f6c180a62 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:33.819680 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8d7a1-b32e-4e46-9db6-9a5f6c180a62 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:33.819954 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8d7a1-b32e-4e46-9db6-9a5f6c180a62 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:33.821713 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 318/616] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 31 14:25:33.827524 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c523f7-8280-4231-803a-0fcd85c4f13b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:33.827811 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c523f7-8280-4231-803a-0fcd85c4f13b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:33.828089 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c523f7-8280-4231-803a-0fcd85c4f13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:33.849397 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ddcb8-51fc-4edb-9b33-189e8867abed None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:33.861408 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-88c523f7-8280-4231-803a-0fcd85c4f13b None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:33.862633 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-88c523f7-8280-4231-803a-0fcd85c4f13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:33.918595 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efa990-5d6f-4a52-946d-d8dda820d252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:33.919456 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efa990-5d6f-4a52-946d-d8dda820d252 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:33.919743 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efa990-5d6f-4a52-946d-d8dda820d252 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:33.920023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efa990-5d6f-4a52-946d-d8dda820d252 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:33.940018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6efa990-5d6f-4a52-946d-d8dda820d252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:33.941484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6efa990-5d6f-4a52-946d-d8dda820d252 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:33.968696 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 319/617] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:33.977381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f6efa990-5d6f-4a52-946d-d8dda820d252 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:33.977381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:33.977381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:33.977381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:33.977381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:33.977381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:33.977381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:33.977381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:33.977381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:33.977381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:33.977381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:33.977381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:33.977381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:33.977381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:33.977381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:33.978470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:33.978470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:33.978470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:33.978470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:33.978470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:33.978470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:33.978470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:33.978470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:33.978470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:33.978470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:33.978470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:33.978470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:33.978470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:33.978470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:33.978470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:33.978470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:33.978470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:33.979625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:33.979625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:33.979625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:33.979625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:33.985394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 299/618] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:33.986121 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c523f7-8280-4231-803a-0fcd85c4f13b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:33.998308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efa990-5d6f-4a52-946d-d8dda820d252 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:34.039081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:34.147176 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efa990-5d6f-4a52-946d-d8dda820d252 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:34.148900 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:34.186756 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c867e4-89d4-457c-8e96-7dfd9c92c3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:34.187570 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c867e4-89d4-457c-8e96-7dfd9c92c3c5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:34.187853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c867e4-89d4-457c-8e96-7dfd9c92c3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:34.188127 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c867e4-89d4-457c-8e96-7dfd9c92c3c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:34.199048 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c867e4-89d4-457c-8e96-7dfd9c92c3c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:34.207090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-b1c867e4-89d4-457c-8e96-7dfd9c92c3c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:34.207090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:34.207090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:34.207090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:34.207090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:34.207090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:34.207090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:34.207090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:34.207090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:34.207090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:34.207090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:34.207090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:34.207090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:34.207090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:34.207090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:34.207090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:34.208017 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:34.208017 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:34.208017 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:34.208017 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:34.208017 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:34.208017 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:34.208017 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:34.208017 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:34.208017 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:34.208017 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:34.208017 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:34.208017 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:34.208017 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:34.208017 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:34.208017 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:34.208017 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:34.208865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:34.208865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:34.208865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:34.208865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:34.208865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:34.208865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:34.208865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:34.208865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:34.208865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:34.208865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:34.208865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:34.208865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:34.208865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 300/619] 10.222.0.67 () {56 vars in 1102 bytes} [Tue May 31 14:25:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:25:34.215181 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c867e4-89d4-457c-8e96-7dfd9c92c3c5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:34.238489 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c523f7-8280-4231-803a-0fcd85c4f13b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:34.310047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f5d707-9dce-4423-a0e4-3645d6ecea26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:34.310740 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f5d707-9dce-4423-a0e4-3645d6ecea26 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:34.311025 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f5d707-9dce-4423-a0e4-3645d6ecea26 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:34.311302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f5d707-9dce-4423-a0e4-3645d6ecea26 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:34.312035 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f5d707-9dce-4423-a0e4-3645d6ecea26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:34.312340 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f5d707-9dce-4423-a0e4-3645d6ecea26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:34.313735 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f5d707-9dce-4423-a0e4-3645d6ecea26 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:34.336432 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 320/620] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:34.344483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f5d707-9dce-4423-a0e4-3645d6ecea26 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:34.412849 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c867e4-89d4-457c-8e96-7dfd9c92c3c5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:34.479342 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389ed536-7580-45e9-8c45-19a351e8009e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:34.480622 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ed536-7580-45e9-8c45-19a351e8009e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:34.480908 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ed536-7580-45e9-8c45-19a351e8009e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:34.481184 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ed536-7580-45e9-8c45-19a351e8009e None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:34.481940 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389ed536-7580-45e9-8c45-19a351e8009e None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:34.482248 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389ed536-7580-45e9-8c45-19a351e8009e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:34.487060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389ed536-7580-45e9-8c45-19a351e8009e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:34.496154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 301/621] 10.222.0.67 () {56 vars in 1103 bytes} [Tue May 31 14:25:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:34.497446 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27583173-2121-4775-88a2-ea4bc9c3ae74 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:34.500621 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:34.502191 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27583173-2121-4775-88a2-ea4bc9c3ae74 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:34.505219 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27583173-2121-4775-88a2-ea4bc9c3ae74 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:34.505219 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 302/622] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:34.505219 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27583173-2121-4775-88a2-ea4bc9c3ae74 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:34.520037 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f5d707-9dce-4423-a0e4-3645d6ecea26 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:34.534791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:34.563094 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e9f80-34fa-4bdc-98d0-d54f42fa2264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:34.563954 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e9f80-34fa-4bdc-98d0-d54f42fa2264 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:34.564240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e9f80-34fa-4bdc-98d0-d54f42fa2264 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:34.564571 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e9f80-34fa-4bdc-98d0-d54f42fa2264 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:34.571885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e9f80-34fa-4bdc-98d0-d54f42fa2264 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:34.573523 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e9f80-34fa-4bdc-98d0-d54f42fa2264 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:34.589983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-b63e9f80-34fa-4bdc-98d0-d54f42fa2264 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:34.589983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:34.589983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:34.589983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:34.589983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:34.589983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:34.589983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:34.589983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:34.589983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:34.589983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:34.589983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:34.589983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:34.589983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:25:34.589983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:25:34.589983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:25:34.592079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:25:34.592079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:34.592079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:34.592079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:34.592079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:34.592079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:34.592079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:34.592079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:34.592079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:34.592079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:34.592079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:34.592079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:34.592079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:34.592079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:34.592079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:34.592079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:34.592079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:34.594224 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:25:34.594224 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:25:34.594224 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:25:34.594224 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:25:34.594224 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:25:34.594224 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:25:34.594224 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:34.594224 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:34.594224 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 321/623] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:34.594224 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e5ff9-3e7e-453e-a1b5-106b67d9527e None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:34.594224 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e5ff9-3e7e-453e-a1b5-106b67d9527e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:34.595972 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e5ff9-3e7e-453e-a1b5-106b67d9527e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:34.613666 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-363e5ff9-3e7e-453e-a1b5-106b67d9527e None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:34.614891 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-363e5ff9-3e7e-453e-a1b5-106b67d9527e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:34.635991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27583173-2121-4775-88a2-ea4bc9c3ae74 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:34.683514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc38ee3-f855-491a-b3c0-02d5a716e56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:34.684542 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc38ee3-f855-491a-b3c0-02d5a716e56a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:34.684832 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc38ee3-f855-491a-b3c0-02d5a716e56a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:34.685111 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc38ee3-f855-491a-b3c0-02d5a716e56a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:34.691051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 322/624] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:34.691912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e5ff9-3e7e-453e-a1b5-106b67d9527e None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:34.698879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc38ee3-f855-491a-b3c0-02d5a716e56a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:34.700915 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc38ee3-f855-491a-b3c0-02d5a716e56a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:34.725504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-9dc38ee3-f855-491a-b3c0-02d5a716e56a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:34.725504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:34.725504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:34.725504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:34.725504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:34.725504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:34.725504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:34.725504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:34.725504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:34.725504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:34.725504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:34.725504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:34.725504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:34.725504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:34.725504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:34.727918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:34.727918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:34.727918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:34.727918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:34.727918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:34.727918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:34.727918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:34.727918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:34.727918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:34.727918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:34.727918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:34.727918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:34.727918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:34.727918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:34.727918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:34.727918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:34.727918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:34.729385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:34.729385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:34.729385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:34.729385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:34.729385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 303/625] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:34.729385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc38ee3-f855-491a-b3c0-02d5a716e56a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:34.790870 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:34.822111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e5ff9-3e7e-453e-a1b5-106b67d9527e None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:34.831444 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:34.862856 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee31a0-44e0-4610-aba9-9d761895ef79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:34.863947 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee31a0-44e0-4610-aba9-9d761895ef79 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:34.864332 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee31a0-44e0-4610-aba9-9d761895ef79 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:34.864732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee31a0-44e0-4610-aba9-9d761895ef79 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:34.865624 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee31a0-44e0-4610-aba9-9d761895ef79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:34.865841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc38ee3-f855-491a-b3c0-02d5a716e56a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:34.866124 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee31a0-44e0-4610-aba9-9d761895ef79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:34.867840 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee31a0-44e0-4610-aba9-9d761895ef79 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:34.886678 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 323/626] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:25:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:34.886678 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee31a0-44e0-4610-aba9-9d761895ef79 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:34.914334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364edaa9-6633-40ef-8970-253d88b3956f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:34.916015 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364edaa9-6633-40ef-8970-253d88b3956f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:34.916623 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364edaa9-6633-40ef-8970-253d88b3956f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:34.917093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364edaa9-6633-40ef-8970-253d88b3956f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:34.930620 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364edaa9-6633-40ef-8970-253d88b3956f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:34.942054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-364edaa9-6633-40ef-8970-253d88b3956f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:34.942054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:34.942054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:34.942054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:34.942054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:34.942054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:34.942054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:34.942054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:34.942054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:34.942054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:34.942054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:34.942054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:34.942054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:34.942054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:34.942054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:34.942054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:34.943445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:34.943445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:34.943445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:34.943445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:34.943445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:34.943445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:34.943445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:34.943445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:34.943445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:34.943445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:34.943445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:34.943445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:34.943445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:34.943445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:34.943445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:34.943445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:34.944276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:34.944276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:34.944276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:34.944276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:34.944276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:34.944276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:34.944276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:34.944276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:34.944276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:34.944276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:34.944276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:34.944276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:34.952802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 304/627] 10.222.0.67 () {56 vars in 1102 bytes} [Tue May 31 14:25:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:34.952802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364edaa9-6633-40ef-8970-253d88b3956f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:35.003291 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee31a0-44e0-4610-aba9-9d761895ef79 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:35.040884 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c1fdd-8ac8-4f5e-9533-9d42ba6f0651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:35.042489 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c1fdd-8ac8-4f5e-9533-9d42ba6f0651 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:35.043168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c1fdd-8ac8-4f5e-9533-9d42ba6f0651 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:35.043698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c1fdd-8ac8-4f5e-9533-9d42ba6f0651 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:35.044612 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c1fdd-8ac8-4f5e-9533-9d42ba6f0651 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:35.045120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c1fdd-8ac8-4f5e-9533-9d42ba6f0651 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:35.046813 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c1fdd-8ac8-4f5e-9533-9d42ba6f0651 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:35.048472 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:35.058322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364edaa9-6633-40ef-8970-253d88b3956f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:35.062420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 324/628] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:35.063563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c1fdd-8ac8-4f5e-9533-9d42ba6f0651 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:35.094435 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d045539-4390-4618-a85b-133d88c68a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:35.095351 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d045539-4390-4618-a85b-133d88c68a92 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:35.095639 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d045539-4390-4618-a85b-133d88c68a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:35.095915 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d045539-4390-4618-a85b-133d88c68a92 None admin] PATH_INFO: `/v3/projects/f965a93a2f674598890bd8cafededb2c/users/dfa2282f957941b98db1d7497da3351b/roles/5956aa478bfc4d03919395e637e61c83` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:35.099398 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:35.112405 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d045539-4390-4618-a85b-133d88c68a92 None admin] RBAC: Authorizing `identity:create_grant(project_id=f965a93a2f674598890bd8cafededb2c, user_id=dfa2282f957941b98db1d7497da3351b, role_id=5956aa478bfc4d03919395e637e61c83)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:35.114504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d045539-4390-4618-a85b-133d88c68a92 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:35.150866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 305/629] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:25:34 2022] PUT /identity/v3/projects/f965a93a2f674598890bd8cafededb2c/users/dfa2282f957941b98db1d7497da3351b/roles/5956aa478bfc4d03919395e637e61c83 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:25:35.150866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d045539-4390-4618-a85b-133d88c68a92 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:35.224334 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c1fdd-8ac8-4f5e-9533-9d42ba6f0651 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:35.293566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c15961-e399-4038-a18d-7b89cb35467a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:35.294261 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c15961-e399-4038-a18d-7b89cb35467a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:35.294541 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c15961-e399-4038-a18d-7b89cb35467a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:35.294849 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c15961-e399-4038-a18d-7b89cb35467a None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:35.295619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c15961-e399-4038-a18d-7b89cb35467a None admin] RBAC: Adding query filter params (name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:35.295928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c15961-e399-4038-a18d-7b89cb35467a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:35.297352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c15961-e399-4038-a18d-7b89cb35467a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:35.314541 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 325/630] 10.222.0.67 () {56 vars in 1103 bytes} [Tue May 31 14:25:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:35.321017 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c15961-e399-4038-a18d-7b89cb35467a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:35.353275 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d045539-4390-4618-a85b-133d88c68a92 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:35.432156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2981a2-3d98-48fe-b972-843f586e503f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:35.434118 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2981a2-3d98-48fe-b972-843f586e503f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:35.434695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2981a2-3d98-48fe-b972-843f586e503f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:35.435142 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2981a2-3d98-48fe-b972-843f586e503f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:35.440283 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:35.447308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2981a2-3d98-48fe-b972-843f586e503f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:35.450182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2981a2-3d98-48fe-b972-843f586e503f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:35.469379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-4c2981a2-3d98-48fe-b972-843f586e503f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 31 14:25:35.469379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:35.469379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:35.469379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:35.469379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:35.469379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:35.469379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:35.469379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:35.469379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:35.469379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:35.469379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:35.469379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:35.469379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:25:35.469379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:25:35.469379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:25:35.469379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:25:35.470573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:35.470573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:35.470573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:25:35.470573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:35.470573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:25:35.470573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:35.470573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:35.470573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:35.470573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:35.470573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:35.470573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:35.470573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:35.470573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:35.470573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:35.470573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:35.470573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:35.470573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:35.470573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:35.471800 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:35.471800 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:35.471800 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:25:35.471800 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:25:35.471800 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:25:35.471800 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:25:35.471800 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:25:35.471800 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:25:35.471800 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 31 14:25:35.471800 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:35.473938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 306/631] 10.222.0.67 () {56 vars in 1093 bytes} [Tue May 31 14:25:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:35.479402 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2981a2-3d98-48fe-b972-843f586e503f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:35.501801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c15961-e399-4038-a18d-7b89cb35467a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:35.538006 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:35.539740 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b0391-19af-41f3-b638-ee7c1a16b2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:35.542173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b0391-19af-41f3-b638-ee7c1a16b2fe None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:35.542173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b0391-19af-41f3-b638-ee7c1a16b2fe None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:35.542173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b0391-19af-41f3-b638-ee7c1a16b2fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:35.553667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b0391-19af-41f3-b638-ee7c1a16b2fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:35.555600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b0391-19af-41f3-b638-ee7c1a16b2fe None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:35.574020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-511b0391-19af-41f3-b638-ee7c1a16b2fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:35.574020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:35.574020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:35.574020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:35.574020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:35.574020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:35.574020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:35.574020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:35.574020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:35.574020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:35.574020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:35.574020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:35.574020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:35.574020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:35.574020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:35.575214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:35.575214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:35.575214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:35.575214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:35.575214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:35.575214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:35.575214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:35.575214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:35.575214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:35.575214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:35.575214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:35.575214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:35.575214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:35.575214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:35.575214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:35.575214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:35.575214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:35.576131 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:35.576131 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:35.576131 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:35.576131 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:35.578521 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 326/632] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:35.586294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b0391-19af-41f3-b638-ee7c1a16b2fe None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:35.651763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2981a2-3d98-48fe-b972-843f586e503f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:35.728941 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3f2f4-bd95-4b78-820f-41d0a3e71fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:35.731831 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3f2f4-bd95-4b78-820f-41d0a3e71fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:35.732939 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3f2f4-bd95-4b78-820f-41d0a3e71fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:35.733216 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3f2f4-bd95-4b78-820f-41d0a3e71fc7 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:35.734065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab3f2f4-bd95-4b78-820f-41d0a3e71fc7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:35.734376 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab3f2f4-bd95-4b78-820f-41d0a3e71fc7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:35.735839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab3f2f4-bd95-4b78-820f-41d0a3e71fc7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:35.755061 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 307/633] 10.222.0.67 () {56 vars in 1133 bytes} [Tue May 31 14:25:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:35.763712 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3f2f4-bd95-4b78-820f-41d0a3e71fc7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:35.774505 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b0391-19af-41f3-b638-ee7c1a16b2fe None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:35.793279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:35.837148 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff13cc-70e3-4657-a95c-d30774277dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:35.837838 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff13cc-70e3-4657-a95c-d30774277dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:35.838123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff13cc-70e3-4657-a95c-d30774277dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:35.838685 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff13cc-70e3-4657-a95c-d30774277dc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:35.840690 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff13cc-70e3-4657-a95c-d30774277dc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:35.841057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff13cc-70e3-4657-a95c-d30774277dc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:35.842471 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff13cc-70e3-4657-a95c-d30774277dc8 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:35.856419 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 327/634] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:35.861861 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aee87c-6133-4e28-8e17-228302d22771 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:35.861861 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aee87c-6133-4e28-8e17-228302d22771 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:35.861861 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aee87c-6133-4e28-8e17-228302d22771 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:35.861861 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 328/635] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:35.862789 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aee87c-6133-4e28-8e17-228302d22771 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:35.882414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3f2f4-bd95-4b78-820f-41d0a3e71fc7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:35.884827 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:35.920286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3c497-0dae-4aab-aee6-0cdd400f9d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:35.920947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3c497-0dae-4aab-aee6-0cdd400f9d69 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:35.921234 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3c497-0dae-4aab-aee6-0cdd400f9d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:35.921512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3c497-0dae-4aab-aee6-0cdd400f9d69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:35.975930 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aee87c-6133-4e28-8e17-228302d22771 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:35.988670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca3c497-0dae-4aab-aee6-0cdd400f9d69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:35.990060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca3c497-0dae-4aab-aee6-0cdd400f9d69 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:36.010947 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb1afd-1259-445d-8e98-f402e7b20a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:36.012300 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb1afd-1259-445d-8e98-f402e7b20a64 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:36.012683 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb1afd-1259-445d-8e98-f402e7b20a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:36.012961 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb1afd-1259-445d-8e98-f402e7b20a64 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:36.021444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb1afd-1259-445d-8e98-f402e7b20a64 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:36.022999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb1afd-1259-445d-8e98-f402e7b20a64 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:36.037895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-8deb1afd-1259-445d-8e98-f402e7b20a64 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 31 14:25:36.037895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:36.037895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:36.037895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:36.037895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:36.037895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:36.037895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:36.037895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:36.037895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:36.037895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:36.037895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:36.037895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:36.037895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:25:36.037895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:25:36.037895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:25:36.037895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:25:36.038924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:36.038924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:36.038924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:25:36.038924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:36.038924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:25:36.038924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:36.038924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:36.038924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:36.038924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:36.038924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:36.038924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:36.038924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:36.038924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:36.038924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:36.038924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:36.038924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:36.038924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:36.038924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:36.039903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:36.039903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:36.039903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:25:36.039903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:25:36.039903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:25:36.039903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:25:36.039903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:25:36.039903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:25:36.039903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 31 14:25:36.039903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:36.039903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 329/636] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:25:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:36.040573 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d7109-c25a-4c9d-ab88-20ca0a4f18e4 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:36.040573 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d7109-c25a-4c9d-ab88-20ca0a4f18e4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:36.040858 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d7109-c25a-4c9d-ab88-20ca0a4f18e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:36.056943 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-616d7109-c25a-4c9d-ab88-20ca0a4f18e4 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:36.058610 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-616d7109-c25a-4c9d-ab88-20ca0a4f18e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:36.083820 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:36.111117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 308/637] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:25:36.111117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee18eea-068f-4a65-8073-a8f55886bc3e None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:36.111117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee18eea-068f-4a65-8073-a8f55886bc3e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:36.111117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee18eea-068f-4a65-8073-a8f55886bc3e None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:36.111117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 309/638] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:36.111117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee18eea-068f-4a65-8073-a8f55886bc3e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:36.114937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:36.146858 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 330/639] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:36.146858 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d7109-c25a-4c9d-ab88-20ca0a4f18e4 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:36.229178 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee18eea-068f-4a65-8073-a8f55886bc3e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:36.242734 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d7109-c25a-4c9d-ab88-20ca0a4f18e4 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:36.269960 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5419fbe-8071-44c5-bb68-153cd59bfddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:36.270872 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5419fbe-8071-44c5-bb68-153cd59bfddf None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:36.271318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5419fbe-8071-44c5-bb68-153cd59bfddf None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:36.271760 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5419fbe-8071-44c5-bb68-153cd59bfddf None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:36.272730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5419fbe-8071-44c5-bb68-153cd59bfddf None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:36.273181 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5419fbe-8071-44c5-bb68-153cd59bfddf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:36.274790 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5419fbe-8071-44c5-bb68-153cd59bfddf None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:36.278930 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12605924-6b8a-4f8d-8004-5146bae2eb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:36.279721 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12605924-6b8a-4f8d-8004-5146bae2eb27 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:36.280001 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12605924-6b8a-4f8d-8004-5146bae2eb27 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:36.280275 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12605924-6b8a-4f8d-8004-5146bae2eb27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:36.290253 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12605924-6b8a-4f8d-8004-5146bae2eb27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:36.291730 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12605924-6b8a-4f8d-8004-5146bae2eb27 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:36.293937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 310/640] 10.222.0.67 () {56 vars in 1137 bytes} [Tue May 31 14:25:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:36.295639 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a7094-be50-4164-97f3-80084747ad2b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:36.296057 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a7094-be50-4164-97f3-80084747ad2b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:36.296489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a7094-be50-4164-97f3-80084747ad2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:36.310114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-12605924-6b8a-4f8d-8004-5146bae2eb27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:36.310114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:36.310114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:36.310114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:36.310114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:36.310114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:36.310114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:36.310114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:36.310114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:36.310114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:36.310114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:36.310114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:36.310114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:36.310114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:36.310114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:36.311240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:36.311240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:36.311240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:36.311240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:36.311240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:36.311240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:36.311240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:36.311240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:36.311240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:36.311240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:36.311240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:36.311240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:36.311240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:36.311240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:36.311240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:36.311240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:36.311240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:36.312151 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:36.312151 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:36.312151 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:36.312151 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:36.312151 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 331/641] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:36.312430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12605924-6b8a-4f8d-8004-5146bae2eb27 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:36.320165 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5d8a7094-be50-4164-97f3-80084747ad2b None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:36.321965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5d8a7094-be50-4164-97f3-80084747ad2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:36.324110 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:36.387798 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:36.399845 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 311/642] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:36.408885 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a7094-be50-4164-97f3-80084747ad2b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:36.415990 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12605924-6b8a-4f8d-8004-5146bae2eb27 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:36.452484 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa903893-3c91-4df7-ab5e-623bcc834dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:36.453296 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa903893-3c91-4df7-ab5e-623bcc834dba None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:36.453583 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa903893-3c91-4df7-ab5e-623bcc834dba None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:36.453860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa903893-3c91-4df7-ab5e-623bcc834dba None admin] PATH_INFO: `/v3/roles/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:36.465986 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa903893-3c91-4df7-ab5e-623bcc834dba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:36.473801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-fa903893-3c91-4df7-ab5e-623bcc834dba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:36.473801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:36.473801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:36.473801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:36.473801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:36.473801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:36.473801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:36.473801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:36.473801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:36.473801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:36.473801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:36.473801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:36.473801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:36.473801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:36.473801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:36.473801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:36.474839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:36.474839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:36.474839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:36.474839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:36.474839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:36.474839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:36.474839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:36.474839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:36.474839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:36.474839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:36.474839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:36.474839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:36.474839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:36.474839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:36.474839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:36.474839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:36.475731 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:36.475731 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:36.475731 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:36.475731 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:36.475731 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:36.475731 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:36.475731 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:36.475731 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:36.475731 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:36.475731 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:36.475731 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:36.475731 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:36.475731 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 332/643] 10.222.0.67 () {56 vars in 1102 bytes} [Tue May 31 14:25:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:36.476442 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa903893-3c91-4df7-ab5e-623bcc834dba None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:36.530891 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a7094-be50-4164-97f3-80084747ad2b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:36.559621 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:36.568412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fd4a7-6208-4ae1-a064-d2b603d139dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:36.569442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fd4a7-6208-4ae1-a064-d2b603d139dc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:36.569729 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fd4a7-6208-4ae1-a064-d2b603d139dc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:36.570005 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fd4a7-6208-4ae1-a064-d2b603d139dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:36.593483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa903893-3c91-4df7-ab5e-623bcc834dba None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:36.629322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:36.630760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e3a06-6d1e-4836-b079-5fd413453fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:36.631436 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e3a06-6d1e-4836-b079-5fd413453fdc None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:36.631793 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e3a06-6d1e-4836-b079-5fd413453fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:36.632080 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e3a06-6d1e-4836-b079-5fd413453fdc None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:36.633561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e3a06-6d1e-4836-b079-5fd413453fdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:36.633882 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e3a06-6d1e-4836-b079-5fd413453fdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:36.635287 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e3a06-6d1e-4836-b079-5fd413453fdc None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:36.640119 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425fd4a7-6208-4ae1-a064-d2b603d139dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:36.641620 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425fd4a7-6208-4ae1-a064-d2b603d139dc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:36.652722 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 333/644] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:36.652722 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e3a06-6d1e-4836-b079-5fd413453fdc None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:36.746209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 312/645] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:25:36.747077 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fd4a7-6208-4ae1-a064-d2b603d139dc None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:36.761869 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e3a06-6d1e-4836-b079-5fd413453fdc None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:36.794534 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:36.801527 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df30fe-85a0-4cb7-8839-2b8c45c5e464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:36.802685 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df30fe-85a0-4cb7-8839-2b8c45c5e464 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:36.803073 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df30fe-85a0-4cb7-8839-2b8c45c5e464 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:36.803453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df30fe-85a0-4cb7-8839-2b8c45c5e464 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:36.814894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47df30fe-85a0-4cb7-8839-2b8c45c5e464 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:36.823123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-47df30fe-85a0-4cb7-8839-2b8c45c5e464 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:36.823123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:36.823123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:36.823123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:36.823123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:36.823123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:36.823123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:36.823123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:36.823123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:36.823123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:36.823123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:36.823123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:36.823123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:36.823123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:36.823123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:36.823123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:36.824309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:36.824309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:36.824309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:36.824309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:36.824309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:36.824309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:36.824309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:36.824309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:36.824309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:36.824309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:36.824309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:36.824309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:36.824309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:36.824309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:36.824309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:36.824309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:36.825457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:36.825457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:36.825457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:36.825457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:36.825457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:36.825457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:36.825457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:36.825457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:36.825457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:36.825457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:36.825457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:36.825457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:36.825457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 334/646] 10.222.0.67 () {56 vars in 1102 bytes} [Tue May 31 14:25:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:36.826597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df30fe-85a0-4cb7-8839-2b8c45c5e464 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:36.841608 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:36.851765 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fd4a7-6208-4ae1-a064-d2b603d139dc None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:36.889697 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4930350-65d2-4f55-a7c2-729f88c4fc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:36.890372 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4930350-65d2-4f55-a7c2-729f88c4fc65 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:36.890674 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4930350-65d2-4f55-a7c2-729f88c4fc65 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:36.890957 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4930350-65d2-4f55-a7c2-729f88c4fc65 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:36.891820 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4930350-65d2-4f55-a7c2-729f88c4fc65 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:36.892245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4930350-65d2-4f55-a7c2-729f88c4fc65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:36.893617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4930350-65d2-4f55-a7c2-729f88c4fc65 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:36.907998 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 313/647] 10.222.0.67 () {56 vars in 1103 bytes} [Tue May 31 14:25:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:36.907998 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4930350-65d2-4f55-a7c2-729f88c4fc65 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:36.933707 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df30fe-85a0-4cb7-8839-2b8c45c5e464 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:36.970214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96293abe-d9d4-4988-8a13-783db1097978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:36.971280 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96293abe-d9d4-4988-8a13-783db1097978 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:36.971570 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96293abe-d9d4-4988-8a13-783db1097978 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:36.971850 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96293abe-d9d4-4988-8a13-783db1097978 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:36.979192 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96293abe-d9d4-4988-8a13-783db1097978 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:36.981197 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96293abe-d9d4-4988-8a13-783db1097978 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:36.995643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-96293abe-d9d4-4988-8a13-783db1097978 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:36.995643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:36.995643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:36.995643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:36.995643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:36.995643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:36.995643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:36.995643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:36.995643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:36.995643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:36.995643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:36.995643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:36.995643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:25:36.995643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:25:36.995643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:25:36.996561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:25:36.996561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:36.996561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:36.996561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:36.996561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:36.996561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:36.996561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:36.996561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:36.996561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:36.996561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:36.996561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:36.996561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:36.996561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:36.996561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:36.996561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:36.996561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:36.996561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:36.997449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:25:36.997449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:25:36.997449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:25:36.997449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:25:36.997449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:25:36.997449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:25:36.997449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:36.997449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:37.001524 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 335/648] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:37.002205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96293abe-d9d4-4988-8a13-783db1097978 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:37.021449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4930350-65d2-4f55-a7c2-729f88c4fc65 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:37.055935 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:37.059755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a1855-265b-45e1-ae5e-a4d296efa563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:37.060845 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a1855-265b-45e1-ae5e-a4d296efa563 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:37.061281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a1855-265b-45e1-ae5e-a4d296efa563 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:37.062224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a1855-265b-45e1-ae5e-a4d296efa563 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:37.074185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87a1855-265b-45e1-ae5e-a4d296efa563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:37.075985 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87a1855-265b-45e1-ae5e-a4d296efa563 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:37.094974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-d87a1855-265b-45e1-ae5e-a4d296efa563 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:37.094974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:37.094974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:37.094974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:37.094974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:37.094974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:37.094974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:37.094974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:37.094974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:37.094974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:37.094974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:37.094974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:37.094974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:37.094974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:37.094974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:37.096047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:37.096047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:37.096047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:37.096047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:37.096047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:37.096047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:37.096047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:37.096047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:37.096047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:37.096047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:37.096047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:37.096047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:37.096047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:37.096047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:37.096047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:37.096047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:37.096047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:37.097136 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:37.097136 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:37.097136 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:37.097136 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:37.098782 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 314/649] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:37.099713 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a1855-265b-45e1-ae5e-a4d296efa563 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:37.110380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:37.111526 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96293abe-d9d4-4988-8a13-783db1097978 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:37.155640 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab1da4-a9e8-4705-88ea-eec44f88bd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:37.156338 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab1da4-a9e8-4705-88ea-eec44f88bd45 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:37.156623 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab1da4-a9e8-4705-88ea-eec44f88bd45 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:37.156904 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab1da4-a9e8-4705-88ea-eec44f88bd45 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:37.157993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab1da4-a9e8-4705-88ea-eec44f88bd45 None admin] RBAC: Adding query filter params (name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:37.158547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab1da4-a9e8-4705-88ea-eec44f88bd45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:37.160750 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab1da4-a9e8-4705-88ea-eec44f88bd45 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:37.170022 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 336/650] 10.222.0.67 () {56 vars in 1103 bytes} [Tue May 31 14:25:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:37.170882 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab1da4-a9e8-4705-88ea-eec44f88bd45 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:37.221191 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a1855-265b-45e1-ae5e-a4d296efa563 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:37.256612 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb277adf-74de-4b68-9501-37773af65fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:37.257411 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb277adf-74de-4b68-9501-37773af65fcf None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:37.257696 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb277adf-74de-4b68-9501-37773af65fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:37.257973 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb277adf-74de-4b68-9501-37773af65fcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:37.270865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb277adf-74de-4b68-9501-37773af65fcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:37.273209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb277adf-74de-4b68-9501-37773af65fcf None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:37.274755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab1da4-a9e8-4705-88ea-eec44f88bd45 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:37.291076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:37.292548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-cb277adf-74de-4b68-9501-37773af65fcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:37.292548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:37.292548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:37.292548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:37.292548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:37.292548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:37.292548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:37.292548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:37.292548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:37.292548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:37.292548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:37.292548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:37.292548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:37.292548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:37.292548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:37.293432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:37.293432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:37.293432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:37.293432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:37.293432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:37.293432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:37.293432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:37.293432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:37.293432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:37.293432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:37.293432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:37.293432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:37.293432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:37.293432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:37.293432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:37.293432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:37.293432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:37.294326 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:37.294326 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:37.294326 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:37.294326 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:37.294326 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 315/651] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:37.300059 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb277adf-74de-4b68-9501-37773af65fcf None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:37.318223 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea8bda-2ff8-4c95-a8f8-4523a3d2c918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:37.318848 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea8bda-2ff8-4c95-a8f8-4523a3d2c918 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:37.319134 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea8bda-2ff8-4c95-a8f8-4523a3d2c918 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:37.319413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea8bda-2ff8-4c95-a8f8-4523a3d2c918 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:37.320204 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea8bda-2ff8-4c95-a8f8-4523a3d2c918 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:37.320516 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea8bda-2ff8-4c95-a8f8-4523a3d2c918 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:37.321952 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea8bda-2ff8-4c95-a8f8-4523a3d2c918 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:37.335156 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 337/652] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:25:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:37.335938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea8bda-2ff8-4c95-a8f8-4523a3d2c918 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:37.373276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:37.418770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb277adf-74de-4b68-9501-37773af65fcf None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:37.437736 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea8bda-2ff8-4c95-a8f8-4523a3d2c918 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:37.454797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51c2b6-5ec5-407c-b751-c86f8a0dced3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:37.455571 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51c2b6-5ec5-407c-b751-c86f8a0dced3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:37.455859 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51c2b6-5ec5-407c-b751-c86f8a0dced3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:37.456144 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51c2b6-5ec5-407c-b751-c86f8a0dced3 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:37.456865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51c2b6-5ec5-407c-b751-c86f8a0dced3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:37.457178 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51c2b6-5ec5-407c-b751-c86f8a0dced3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:37.458553 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51c2b6-5ec5-407c-b751-c86f8a0dced3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:37.471228 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 316/653] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:37.473025 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7977f39c-cad3-42e0-94a6-81a3ccfc5569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:37.473829 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977f39c-cad3-42e0-94a6-81a3ccfc5569 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:37.474105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977f39c-cad3-42e0-94a6-81a3ccfc5569 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:37.474376 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977f39c-cad3-42e0-94a6-81a3ccfc5569 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:37.476297 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51c2b6-5ec5-407c-b751-c86f8a0dced3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:37.485231 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7977f39c-cad3-42e0-94a6-81a3ccfc5569 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:37.486730 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7977f39c-cad3-42e0-94a6-81a3ccfc5569 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:37.505645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-7977f39c-cad3-42e0-94a6-81a3ccfc5569 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:37.505645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:37.505645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:37.505645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:37.505645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:37.505645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:37.505645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:37.505645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:37.505645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:37.505645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:37.505645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:37.505645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:37.505645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:37.505645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:37.505645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:37.506617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:37.506617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:37.506617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:37.506617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:37.506617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:37.506617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:37.506617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:37.506617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:37.506617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:37.506617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:37.506617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:37.506617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:37.506617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:37.506617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:37.506617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:37.506617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:37.506617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:37.507536 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:37.507536 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:37.507536 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:37.507536 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:37.507536 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 338/654] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:37.507824 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7977f39c-cad3-42e0-94a6-81a3ccfc5569 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:37.536991 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:37.589683 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51c2b6-5ec5-407c-b751-c86f8a0dced3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:37.607728 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:37.620020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7977f39c-cad3-42e0-94a6-81a3ccfc5569 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:37.626663 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb6eb4-586b-44d4-9025-c0d0af77e483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:37.627337 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb6eb4-586b-44d4-9025-c0d0af77e483 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:37.627621 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb6eb4-586b-44d4-9025-c0d0af77e483 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:37.627902 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb6eb4-586b-44d4-9025-c0d0af77e483 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:37.628620 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb6eb4-586b-44d4-9025-c0d0af77e483 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:37.628930 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb6eb4-586b-44d4-9025-c0d0af77e483 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:37.630316 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb6eb4-586b-44d4-9025-c0d0af77e483 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:37.642165 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 317/655] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:37.642973 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb6eb4-586b-44d4-9025-c0d0af77e483 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:37.655069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf8292-351d-4abb-8d20-ca7be0164011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:37.655728 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf8292-351d-4abb-8d20-ca7be0164011 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:37.656017 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf8292-351d-4abb-8d20-ca7be0164011 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:37.656297 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf8292-351d-4abb-8d20-ca7be0164011 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:37.657130 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf8292-351d-4abb-8d20-ca7be0164011 None admin] RBAC: Adding query filter params (role.id=5956aa478bfc4d03919395e637e61c83, scope.project.id=f965a93a2f674598890bd8cafededb2c, user.id=68c6363b60954c3ebf50d2019f5f2fd2) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:37.657450 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf8292-351d-4abb-8d20-ca7be0164011 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:37.658901 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf8292-351d-4abb-8d20-ca7be0164011 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:37.664720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 339/656] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:25:37 2022] GET /identity/v3/role_assignments?user.id=68c6363b60954c3ebf50d2019f5f2fd2&scope.project.id=f965a93a2f674598890bd8cafededb2c&role.id=5956aa478bfc4d03919395e637e61c83 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:37.665369 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf8292-351d-4abb-8d20-ca7be0164011 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:37.754088 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb6eb4-586b-44d4-9025-c0d0af77e483 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:37.767793 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:37.781923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf8292-351d-4abb-8d20-ca7be0164011 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:37.790814 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be0792-fcc8-4256-9c5e-e70fa0f5f3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:37.791637 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be0792-fcc8-4256-9c5e-e70fa0f5f3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:37.791921 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be0792-fcc8-4256-9c5e-e70fa0f5f3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:37.792718 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be0792-fcc8-4256-9c5e-e70fa0f5f3c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:37.800688 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be0792-fcc8-4256-9c5e-e70fa0f5f3c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:37.802178 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be0792-fcc8-4256-9c5e-e70fa0f5f3c2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:37.818767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-36be0792-fcc8-4256-9c5e-e70fa0f5f3c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:37.818767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:37.818767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:37.818767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:37.818767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:37.818767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:37.818767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:37.818767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:37.818767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:37.818767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:37.818767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:37.818767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:37.818767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:25:37.818767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:25:37.818767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:25:37.819928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:25:37.819928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:37.819928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:37.819928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:37.819928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:37.819928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:37.819928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:37.819928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:37.819928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:37.819928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:37.819928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:37.819928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:37.819928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:37.819928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:37.819928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:37.819928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:37.819928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:37.820917 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:25:37.820917 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:25:37.820917 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:25:37.820917 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:25:37.820917 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:25:37.820917 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:25:37.820917 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:37.820917 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:37.820917 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 318/657] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:37.822203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87d17d-dbb6-453e-8dd3-044d47ceceb0 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:37.822493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87d17d-dbb6-453e-8dd3-044d47ceceb0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:37.823176 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87d17d-dbb6-453e-8dd3-044d47ceceb0 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:37.824672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 319/658] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:37.827597 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a87d17d-dbb6-453e-8dd3-044d47ceceb0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:37.830221 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b3e22-c343-4016-ac92-f24df1c63eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:37.831089 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b3e22-c343-4016-ac92-f24df1c63eca None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:37.831379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b3e22-c343-4016-ac92-f24df1c63eca None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:37.831786 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b3e22-c343-4016-ac92-f24df1c63eca None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:37.832537 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b3e22-c343-4016-ac92-f24df1c63eca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:37.832879 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b3e22-c343-4016-ac92-f24df1c63eca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:37.834789 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b3e22-c343-4016-ac92-f24df1c63eca None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:37.853024 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 340/659] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:37.853024 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b3e22-c343-4016-ac92-f24df1c63eca None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:37.880121 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:37.959043 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a87d17d-dbb6-453e-8dd3-044d47ceceb0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:37.970449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b3e22-c343-4016-ac92-f24df1c63eca None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:37.994271 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d6c0a-dd04-4c0c-b098-288f9563c79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:37.995123 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d6c0a-dd04-4c0c-b098-288f9563c79e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:37.995976 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d6c0a-dd04-4c0c-b098-288f9563c79e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:37.996267 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d6c0a-dd04-4c0c-b098-288f9563c79e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:38.003479 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01d6c0a-dd04-4c0c-b098-288f9563c79e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:38.004912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01d6c0a-dd04-4c0c-b098-288f9563c79e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:38.007849 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a597f-57b1-4a9c-ab72-63cbfdc75c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:38.008702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a597f-57b1-4a9c-ab72-63cbfdc75c75 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:38.009155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a597f-57b1-4a9c-ab72-63cbfdc75c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:38.009601 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a597f-57b1-4a9c-ab72-63cbfdc75c75 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:38.010576 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a597f-57b1-4a9c-ab72-63cbfdc75c75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:38.011082 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a597f-57b1-4a9c-ab72-63cbfdc75c75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:38.012633 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a597f-57b1-4a9c-ab72-63cbfdc75c75 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:38.019431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e01d6c0a-dd04-4c0c-b098-288f9563c79e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 31 14:25:38.019431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:38.019431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:38.019431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:38.019431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:38.019431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:38.019431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:38.019431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:38.019431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:38.019431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:38.019431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:38.019431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:38.019431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:25:38.019431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:25:38.019431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:25:38.019431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:25:38.020543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:38.020543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:38.020543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:25:38.020543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:38.020543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:25:38.020543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:38.020543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:38.020543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:38.020543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:38.020543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:38.020543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:38.020543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:38.020543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:38.020543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:38.020543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:38.020543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:38.020543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:38.020543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:38.021496 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:38.021496 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:38.021496 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:25:38.021496 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:25:38.021496 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:25:38.021496 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:25:38.021496 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:25:38.021496 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:25:38.021496 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 31 14:25:38.021496 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:38.021496 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 320/660] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:25:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:38.022518 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66858b78-9d45-4fa1-8da1-976a90fdc814 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:38.022518 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66858b78-9d45-4fa1-8da1-976a90fdc814 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:38.022710 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66858b78-9d45-4fa1-8da1-976a90fdc814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:38.031574 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 341/661] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:25:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:38.032595 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a597f-57b1-4a9c-ab72-63cbfdc75c75 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:38.041396 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-66858b78-9d45-4fa1-8da1-976a90fdc814 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:38.043198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-66858b78-9d45-4fa1-8da1-976a90fdc814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:38.044665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:38.111805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 321/662] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:38.112487 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66858b78-9d45-4fa1-8da1-976a90fdc814 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:38.147139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:38.162363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a597f-57b1-4a9c-ab72-63cbfdc75c75 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:38.199910 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dab3ed-bdff-4aa1-a262-7e12d5627e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:38.200880 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dab3ed-bdff-4aa1-a262-7e12d5627e3c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:38.201019 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dab3ed-bdff-4aa1-a262-7e12d5627e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:38.201495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dab3ed-bdff-4aa1-a262-7e12d5627e3c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:38.208809 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dab3ed-bdff-4aa1-a262-7e12d5627e3c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:38.210259 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dab3ed-bdff-4aa1-a262-7e12d5627e3c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:38.225401 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66858b78-9d45-4fa1-8da1-976a90fdc814 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:38.225577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-37dab3ed-bdff-4aa1-a262-7e12d5627e3c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 31 14:25:38.225577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:38.225577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:38.225577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:38.225577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:38.225577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:38.225577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:38.225577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:38.225577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:38.225577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:38.225577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:38.225577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:38.225577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:25:38.225577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:25:38.225577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:25:38.225577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:25:38.226444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:38.226444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:38.226444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:25:38.226444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:38.226444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:25:38.226444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:38.226444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:38.226444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:38.226444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:38.226444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:38.226444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:38.226444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:38.226444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:38.226444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:38.226444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:38.226444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:38.226444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:38.226444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:38.229373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:38.229373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:38.229373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:25:38.229373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:25:38.229373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:25:38.229373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:25:38.229373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:25:38.229373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:25:38.229373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 31 14:25:38.229373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:38.231053 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 342/663] 10.222.0.67 () {56 vars in 1102 bytes} [Tue May 31 14:25:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:38.231570 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dab3ed-bdff-4aa1-a262-7e12d5627e3c None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:38.267053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46889137-4747-43b5-9686-20bdcc514e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:38.267738 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46889137-4747-43b5-9686-20bdcc514e52 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:38.268025 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46889137-4747-43b5-9686-20bdcc514e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:38.268308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46889137-4747-43b5-9686-20bdcc514e52 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:38.269150 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46889137-4747-43b5-9686-20bdcc514e52 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:38.269467 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46889137-4747-43b5-9686-20bdcc514e52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:38.270923 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46889137-4747-43b5-9686-20bdcc514e52 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:38.290246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 322/664] 10.222.0.67 () {56 vars in 1137 bytes} [Tue May 31 14:25:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:38.290985 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46889137-4747-43b5-9686-20bdcc514e52 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:38.305856 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:38.341790 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dab3ed-bdff-4aa1-a262-7e12d5627e3c None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:38.376768 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659792a-0f65-4f17-9e80-91d78ac074ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:38.377489 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659792a-0f65-4f17-9e80-91d78ac074ce None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:38.377774 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659792a-0f65-4f17-9e80-91d78ac074ce None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:38.378051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659792a-0f65-4f17-9e80-91d78ac074ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:38.378904 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659792a-0f65-4f17-9e80-91d78ac074ce None admin] RBAC: Adding query filter params (role.id=5956aa478bfc4d03919395e637e61c83, scope.project.id=f965a93a2f674598890bd8cafededb2c, user.id=d8a4b0dd4e21414d91fe7bed478d15e7) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:38.379222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659792a-0f65-4f17-9e80-91d78ac074ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:38.380649 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659792a-0f65-4f17-9e80-91d78ac074ce None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:38.398687 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 343/665] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:25:38 2022] GET /identity/v3/role_assignments?user.id=d8a4b0dd4e21414d91fe7bed478d15e7&scope.project.id=f965a93a2f674598890bd8cafededb2c&role.id=5956aa478bfc4d03919395e637e61c83 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:38.398687 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659792a-0f65-4f17-9e80-91d78ac074ce None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:38.398978 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:38.407246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46889137-4747-43b5-9686-20bdcc514e52 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:38.447442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efad056-03af-44b6-8505-7f2d2db5cc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:38.448406 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efad056-03af-44b6-8505-7f2d2db5cc56 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:38.448771 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efad056-03af-44b6-8505-7f2d2db5cc56 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:38.449122 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efad056-03af-44b6-8505-7f2d2db5cc56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:38.462810 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efad056-03af-44b6-8505-7f2d2db5cc56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:38.471731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-4efad056-03af-44b6-8505-7f2d2db5cc56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:38.471731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:38.471731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:38.471731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:38.471731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:38.471731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:38.471731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:38.471731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:38.471731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:38.471731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:38.471731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:38.471731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:38.471731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:38.471731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:38.471731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:38.471731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:38.472922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:38.472922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:38.472922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:38.472922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:38.472922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:38.472922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:38.472922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:38.472922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:38.472922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:38.472922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:38.472922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:38.472922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:38.472922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:38.472922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:38.472922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:38.472922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:38.473803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:38.473803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:38.473803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:38.473803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:38.473803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:38.473803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:38.473803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:38.473803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:38.473803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:38.473803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:38.473803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:38.473803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:38.483238 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 323/666] 10.222.0.67 () {56 vars in 1102 bytes} [Tue May 31 14:25:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:38.484567 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efad056-03af-44b6-8505-7f2d2db5cc56 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:38.538944 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659792a-0f65-4f17-9e80-91d78ac074ce None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:38.577316 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a8010-60e7-4e0f-8eed-9cacb9a31e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:38.578105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a8010-60e7-4e0f-8eed-9cacb9a31e87 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:38.578465 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a8010-60e7-4e0f-8eed-9cacb9a31e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:38.578861 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a8010-60e7-4e0f-8eed-9cacb9a31e87 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:38.579787 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a8010-60e7-4e0f-8eed-9cacb9a31e87 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:38.580227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a8010-60e7-4e0f-8eed-9cacb9a31e87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:38.581803 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a8010-60e7-4e0f-8eed-9cacb9a31e87 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:38.597531 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:38.603649 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 344/667] 10.222.0.67 () {56 vars in 1139 bytes} [Tue May 31 14:25:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:38.604501 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a8010-60e7-4e0f-8eed-9cacb9a31e87 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:38.613852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efad056-03af-44b6-8505-7f2d2db5cc56 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:38.657441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80130cc-9840-44bc-b581-d98f45952d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:38.658015 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80130cc-9840-44bc-b581-d98f45952d26 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:38.658305 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80130cc-9840-44bc-b581-d98f45952d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:38.658578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80130cc-9840-44bc-b581-d98f45952d26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:38.678866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:38.730944 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a8010-60e7-4e0f-8eed-9cacb9a31e87 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:38.753716 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80130cc-9840-44bc-b581-d98f45952d26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:38.755492 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80130cc-9840-44bc-b581-d98f45952d26 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:38.773986 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752493c-fc75-4bec-9da4-c27995a2945e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:38.775358 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752493c-fc75-4bec-9da4-c27995a2945e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:38.775821 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752493c-fc75-4bec-9da4-c27995a2945e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:38.776292 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752493c-fc75-4bec-9da4-c27995a2945e None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:38.777788 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752493c-fc75-4bec-9da4-c27995a2945e None admin] RBAC: Adding query filter params (name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:38.778733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752493c-fc75-4bec-9da4-c27995a2945e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:38.780156 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752493c-fc75-4bec-9da4-c27995a2945e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:38.792522 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 345/668] 10.222.0.67 () {56 vars in 1103 bytes} [Tue May 31 14:25:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:38.793739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752493c-fc75-4bec-9da4-c27995a2945e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:38.867096 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:38.874561 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 324/669] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:25:38.875500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80130cc-9840-44bc-b581-d98f45952d26 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:38.920178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752493c-fc75-4bec-9da4-c27995a2945e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:38.924058 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:38.958825 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3946f8c-62bf-4d59-95cf-fd5cf7b6dbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:38.959482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3946f8c-62bf-4d59-95cf-fd5cf7b6dbfc None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:38.959765 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3946f8c-62bf-4d59-95cf-fd5cf7b6dbfc None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:38.960043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3946f8c-62bf-4d59-95cf-fd5cf7b6dbfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:38.981457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80130cc-9840-44bc-b581-d98f45952d26 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:39.020807 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c034a-a830-4bea-9e12-d55f86d546d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:39.021958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c034a-a830-4bea-9e12-d55f86d546d4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:39.022498 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c034a-a830-4bea-9e12-d55f86d546d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:39.022498 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c034a-a830-4bea-9e12-d55f86d546d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:39.036379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c034a-a830-4bea-9e12-d55f86d546d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:39.038174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c034a-a830-4bea-9e12-d55f86d546d4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:39.047258 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3946f8c-62bf-4d59-95cf-fd5cf7b6dbfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:39.049341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3946f8c-62bf-4d59-95cf-fd5cf7b6dbfc None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:39.058569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-390c034a-a830-4bea-9e12-d55f86d546d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:39.058569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:39.058569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:39.058569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:39.058569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:39.058569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:39.058569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:39.058569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:39.058569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:39.058569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:39.058569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:39.058569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:39.058569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:39.058569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:39.058569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:39.059500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:39.059500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:39.059500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:39.059500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:39.059500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:39.059500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:39.059500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:39.059500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:39.059500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:39.059500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:39.059500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:39.059500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:39.059500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:39.059500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:39.059500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:39.059500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:39.059500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:39.060377 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:39.060377 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:39.060377 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:39.060377 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:39.064535 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 325/670] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:39.065253 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c034a-a830-4bea-9e12-d55f86d546d4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:39.110129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:39.170876 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 346/671] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:25:39.176270 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3946f8c-62bf-4d59-95cf-fd5cf7b6dbfc None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:39.208641 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:39.214456 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c034a-a830-4bea-9e12-d55f86d546d4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:39.259869 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287544b-ee71-45ca-8542-b86751d7847b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:39.260856 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287544b-ee71-45ca-8542-b86751d7847b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:39.261437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287544b-ee71-45ca-8542-b86751d7847b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:39.261653 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287544b-ee71-45ca-8542-b86751d7847b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:39.272503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287544b-ee71-45ca-8542-b86751d7847b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:39.274149 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287544b-ee71-45ca-8542-b86751d7847b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:39.287526 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3946f8c-62bf-4d59-95cf-fd5cf7b6dbfc None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:39.293346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-d287544b-ee71-45ca-8542-b86751d7847b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:39.293346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:39.293346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:39.293346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:39.293346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:39.293346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:39.293346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:39.293346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:39.293346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:39.293346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:39.293346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:39.293346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:39.293346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:39.293346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:39.293346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:39.294395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:39.294395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:39.294395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:39.294395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:39.294395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:39.294395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:39.294395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:39.294395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:39.294395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:39.294395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:39.294395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:39.294395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:39.294395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:39.294395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:39.294395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:39.294395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:39.294395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:39.295597 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:39.295597 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:39.295597 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:39.295597 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:39.301722 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 326/672] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:39.301722 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287544b-ee71-45ca-8542-b86751d7847b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:39.337466 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5cdac-f8c8-4b68-b8df-9fb617fe5359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:39.338136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5cdac-f8c8-4b68-b8df-9fb617fe5359 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:39.338424 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5cdac-f8c8-4b68-b8df-9fb617fe5359 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:39.338731 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5cdac-f8c8-4b68-b8df-9fb617fe5359 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:39.339675 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5cdac-f8c8-4b68-b8df-9fb617fe5359 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:39.339995 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5cdac-f8c8-4b68-b8df-9fb617fe5359 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:39.342204 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5cdac-f8c8-4b68-b8df-9fb617fe5359 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:39.361512 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 347/673] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:39.361512 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5cdac-f8c8-4b68-b8df-9fb617fe5359 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:39.378003 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:39.430215 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287544b-ee71-45ca-8542-b86751d7847b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:39.473821 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda253c3-8fbd-4e80-b5d8-498d94ec5493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:39.474830 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda253c3-8fbd-4e80-b5d8-498d94ec5493 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:39.475217 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda253c3-8fbd-4e80-b5d8-498d94ec5493 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:39.475602 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda253c3-8fbd-4e80-b5d8-498d94ec5493 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:39.486732 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda253c3-8fbd-4e80-b5d8-498d94ec5493 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:39.488386 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda253c3-8fbd-4e80-b5d8-498d94ec5493 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:39.491779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5cdac-f8c8-4b68-b8df-9fb617fe5359 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:39.510074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-bda253c3-8fbd-4e80-b5d8-498d94ec5493 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:39.510074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:39.510074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:39.510074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:39.510074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:39.510074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:39.510074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:39.510074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:39.510074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:39.510074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:39.510074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:39.510074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:39.510074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:39.510074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:39.510074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:39.511785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:39.511785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:39.511785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:39.511785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:39.511785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:39.511785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:39.511785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:39.511785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:39.511785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:39.511785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:39.511785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:39.511785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:39.511785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:39.511785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:39.511785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:39.511785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:39.511785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:39.513094 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:39.513094 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:39.513094 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:39.513094 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:39.519028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 327/674] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:39.519028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda253c3-8fbd-4e80-b5d8-498d94ec5493 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:39.522379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:39.541659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d342d2-8b49-4ad1-8fe9-8d36c56879e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:39.542364 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d342d2-8b49-4ad1-8fe9-8d36c56879e7 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:39.542638 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d342d2-8b49-4ad1-8fe9-8d36c56879e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:39.542996 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d342d2-8b49-4ad1-8fe9-8d36c56879e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:39.543733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d342d2-8b49-4ad1-8fe9-8d36c56879e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:39.544046 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d342d2-8b49-4ad1-8fe9-8d36c56879e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:39.545637 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d342d2-8b49-4ad1-8fe9-8d36c56879e7 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:39.563105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 348/675] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:39.563105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d342d2-8b49-4ad1-8fe9-8d36c56879e7 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:39.644088 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda253c3-8fbd-4e80-b5d8-498d94ec5493 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:39.656472 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:39.686011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a1ce2-e049-477f-9a5b-537db8bafe50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:39.686989 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a1ce2-e049-477f-9a5b-537db8bafe50 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:39.687893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a1ce2-e049-477f-9a5b-537db8bafe50 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:39.687893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a1ce2-e049-477f-9a5b-537db8bafe50 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:39.696381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a1ce2-e049-477f-9a5b-537db8bafe50 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:39.698123 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a1ce2-e049-477f-9a5b-537db8bafe50 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:39.704216 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d342d2-8b49-4ad1-8fe9-8d36c56879e7 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:39.714545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-1b0a1ce2-e049-477f-9a5b-537db8bafe50 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 31 14:25:39.714545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:39.714545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:39.714545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:39.714545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:39.714545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:39.714545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:39.714545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:39.714545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:39.714545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:39.714545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:39.714545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:39.714545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:25:39.714545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:25:39.714545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:25:39.714545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:25:39.715717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:39.715717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:39.715717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:25:39.715717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:39.715717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:25:39.715717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:39.715717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:39.715717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:39.715717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:39.715717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:39.715717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:39.715717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:39.715717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:39.715717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:39.715717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:39.715717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:39.715717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:39.715717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:39.716874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:39.716874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:39.716874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:25:39.716874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:25:39.716874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:25:39.716874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:25:39.716874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:25:39.716874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:25:39.716874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 31 14:25:39.716874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:39.723021 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 328/676] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:25:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:39.723021 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a1ce2-e049-477f-9a5b-537db8bafe50 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:39.750137 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb1790f-4880-4c9f-a486-b59ed7725040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:39.750913 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1790f-4880-4c9f-a486-b59ed7725040 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:39.751201 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1790f-4880-4c9f-a486-b59ed7725040 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:39.751480 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1790f-4880-4c9f-a486-b59ed7725040 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:39.752222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb1790f-4880-4c9f-a486-b59ed7725040 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:39.752532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb1790f-4880-4c9f-a486-b59ed7725040 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:39.754074 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb1790f-4880-4c9f-a486-b59ed7725040 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:39.773962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 349/677] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:39.773962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb1790f-4880-4c9f-a486-b59ed7725040 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:39.812947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:39.865431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a1ce2-e049-477f-9a5b-537db8bafe50 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:39.909830 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb1790f-4880-4c9f-a486-b59ed7725040 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:39.911162 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee229f4-d876-42d2-a67b-ade2753c916d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:39.913261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee229f4-d876-42d2-a67b-ade2753c916d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:39.913749 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee229f4-d876-42d2-a67b-ade2753c916d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:39.914221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee229f4-d876-42d2-a67b-ade2753c916d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:39.923617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee229f4-d876-42d2-a67b-ade2753c916d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:39.926099 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee229f4-d876-42d2-a67b-ade2753c916d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:39.940963 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:39.946537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-2ee229f4-d876-42d2-a67b-ade2753c916d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:39.946537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:39.946537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:39.946537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:39.946537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:39.946537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:39.946537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:39.946537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:39.946537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:39.946537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:39.946537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:39.946537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:39.946537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:25:39.946537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:25:39.946537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:25:39.947431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:25:39.947431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:39.947431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:39.947431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:39.947431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:39.947431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:39.947431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:39.947431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:39.947431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:39.947431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:39.947431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:39.947431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:39.947431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:39.947431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:39.947431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:39.947431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:39.947431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:39.948308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:25:39.948308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:25:39.948308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:25:39.948308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:25:39.948308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:25:39.948308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:25:39.948308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:39.948308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:39.949284 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58105ee5-1409-40f4-80f8-2245a0a19ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:39.949284 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58105ee5-1409-40f4-80f8-2245a0a19ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:39.949284 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58105ee5-1409-40f4-80f8-2245a0a19ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:39.949284 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58105ee5-1409-40f4-80f8-2245a0a19ef4 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:39.950036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58105ee5-1409-40f4-80f8-2245a0a19ef4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:39.950036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58105ee5-1409-40f4-80f8-2245a0a19ef4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:39.951789 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58105ee5-1409-40f4-80f8-2245a0a19ef4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:39.957236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 329/678] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:39.957236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee229f4-d876-42d2-a67b-ade2753c916d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:39.980950 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 350/679] 10.222.0.67 () {56 vars in 1135 bytes} [Tue May 31 14:25:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:39.980950 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58105ee5-1409-40f4-80f8-2245a0a19ef4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:40.075586 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee229f4-d876-42d2-a67b-ade2753c916d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:40.090584 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58105ee5-1409-40f4-80f8-2245a0a19ef4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:40.098836 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:40.112840 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab209d-0a32-4027-b188-b6b7b7b3e608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:40.113641 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab209d-0a32-4027-b188-b6b7b7b3e608 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:40.113928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab209d-0a32-4027-b188-b6b7b7b3e608 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:40.114205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab209d-0a32-4027-b188-b6b7b7b3e608 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:40.121760 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ab209d-0a32-4027-b188-b6b7b7b3e608 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:40.123241 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ab209d-0a32-4027-b188-b6b7b7b3e608 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:40.133037 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a982c-11d4-44a0-afb6-7469e6c7cb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:40.133734 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a982c-11d4-44a0-afb6-7469e6c7cb87 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:40.134023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a982c-11d4-44a0-afb6-7469e6c7cb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:40.134304 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a982c-11d4-44a0-afb6-7469e6c7cb87 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:40.135134 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a982c-11d4-44a0-afb6-7469e6c7cb87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:40.135449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a982c-11d4-44a0-afb6-7469e6c7cb87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:40.136920 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a982c-11d4-44a0-afb6-7469e6c7cb87 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:40.138855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-58ab209d-0a32-4027-b188-b6b7b7b3e608 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:40.138855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:40.138855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:40.138855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:40.138855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:40.138855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:40.138855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:40.138855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:40.138855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:40.138855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:40.138855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:40.138855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:40.138855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:25:40.138855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:25:40.138855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:25:40.139709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:25:40.139709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:40.139709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:40.139709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:40.139709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:40.139709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:40.139709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:40.139709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:40.139709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:40.139709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:40.139709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:40.139709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:40.139709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:40.139709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:40.139709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:40.139709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:40.139709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:40.145523 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:25:40.145523 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:25:40.145523 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:25:40.145523 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:25:40.145523 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:25:40.145523 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:25:40.145523 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:40.145523 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:40.145523 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 330/680] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:40.145523 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab209d-0a32-4027-b188-b6b7b7b3e608 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:40.152019 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 351/681] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:25:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:40.159268 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350826e-86a2-4fd9-8a39-944311e0dcef None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:40.159268 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350826e-86a2-4fd9-8a39-944311e0dcef None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:40.159268 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350826e-86a2-4fd9-8a39-944311e0dcef None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:40.159268 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 352/682] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:40.159268 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350826e-86a2-4fd9-8a39-944311e0dcef None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:40.218065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:40.260552 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab209d-0a32-4027-b188-b6b7b7b3e608 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:40.273329 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350826e-86a2-4fd9-8a39-944311e0dcef None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:40.295558 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7f472-d5e5-4be2-a5a2-13116fad4048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:40.296209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7f472-d5e5-4be2-a5a2-13116fad4048 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:40.296493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7f472-d5e5-4be2-a5a2-13116fad4048 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:40.296774 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7f472-d5e5-4be2-a5a2-13116fad4048 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:40.310456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a894fd-84e8-4ece-af48-cf08a63c94dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:40.311151 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a894fd-84e8-4ece-af48-cf08a63c94dd None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:40.311436 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a894fd-84e8-4ece-af48-cf08a63c94dd None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:40.311712 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a894fd-84e8-4ece-af48-cf08a63c94dd None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:40.312498 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a894fd-84e8-4ece-af48-cf08a63c94dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:40.312809 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a894fd-84e8-4ece-af48-cf08a63c94dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:40.314231 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a894fd-84e8-4ece-af48-cf08a63c94dd None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:40.331106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 353/683] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:25:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:40.331106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce69ddf-55c9-4cca-a3cb-7173a44105f4 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:40.331106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce69ddf-55c9-4cca-a3cb-7173a44105f4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:40.331106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce69ddf-55c9-4cca-a3cb-7173a44105f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:40.345232 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6ce69ddf-55c9-4cca-a3cb-7173a44105f4 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:40.346443 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6ce69ddf-55c9-4cca-a3cb-7173a44105f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:40.355984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:40.377202 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7f472-d5e5-4be2-a5a2-13116fad4048 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:40.378665 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7f472-d5e5-4be2-a5a2-13116fad4048 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:40.414093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 354/684] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:40.415537 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce69ddf-55c9-4cca-a3cb-7173a44105f4 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:40.463773 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:40.486242 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 331/685] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:25:40.487405 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7f472-d5e5-4be2-a5a2-13116fad4048 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:40.537383 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce69ddf-55c9-4cca-a3cb-7173a44105f4 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:40.577404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:40.578665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf8e02-d713-4e32-9021-b49f42dde072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:40.579507 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf8e02-d713-4e32-9021-b49f42dde072 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:40.579751 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf8e02-d713-4e32-9021-b49f42dde072 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:40.580072 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf8e02-d713-4e32-9021-b49f42dde072 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:40.581545 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf8e02-d713-4e32-9021-b49f42dde072 None admin] RBAC: Adding query filter params (role.id=5956aa478bfc4d03919395e637e61c83, scope.project.id=f965a93a2f674598890bd8cafededb2c, user.id=990999672d1a4f19896a143e1eeddd44) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:40.581851 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf8e02-d713-4e32-9021-b49f42dde072 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:40.583329 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf8e02-d713-4e32-9021-b49f42dde072 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:40.590574 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 355/686] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:25:40 2022] GET /identity/v3/role_assignments?user.id=990999672d1a4f19896a143e1eeddd44&scope.project.id=f965a93a2f674598890bd8cafededb2c&role.id=5956aa478bfc4d03919395e637e61c83 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:40.591456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf8e02-d713-4e32-9021-b49f42dde072 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:40.611026 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7f472-d5e5-4be2-a5a2-13116fad4048 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:40.656599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b86bab-5788-4868-ac78-a736ba59fb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:40.657294 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b86bab-5788-4868-ac78-a736ba59fb04 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:40.657584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b86bab-5788-4868-ac78-a736ba59fb04 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:40.657867 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b86bab-5788-4868-ac78-a736ba59fb04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:40.658757 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b86bab-5788-4868-ac78-a736ba59fb04 None admin] RBAC: Adding query filter params (role.id=5956aa478bfc4d03919395e637e61c83, scope.project.id=f965a93a2f674598890bd8cafededb2c, user.id=8966bc2ea7ff497c84214c9c9e6d53a8) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:40.659135 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b86bab-5788-4868-ac78-a736ba59fb04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:40.661065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b86bab-5788-4868-ac78-a736ba59fb04 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:40.674720 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 332/687] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:25:40 2022] GET /identity/v3/role_assignments?user.id=8966bc2ea7ff497c84214c9c9e6d53a8&scope.project.id=f965a93a2f674598890bd8cafededb2c&role.id=5956aa478bfc4d03919395e637e61c83 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:40.674720 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b86bab-5788-4868-ac78-a736ba59fb04 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:40.785178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf8e02-d713-4e32-9021-b49f42dde072 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:40.791938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:40.848153 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123648c-d7c4-429d-95d3-78760ab62af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:40.848965 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123648c-d7c4-429d-95d3-78760ab62af4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:40.849244 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123648c-d7c4-429d-95d3-78760ab62af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:40.849668 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123648c-d7c4-429d-95d3-78760ab62af4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:40.862813 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123648c-d7c4-429d-95d3-78760ab62af4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:40.866286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b86bab-5788-4868-ac78-a736ba59fb04 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:40.873298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-e123648c-d7c4-429d-95d3-78760ab62af4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:25:40.873298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:40.873298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:40.873298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:40.873298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:40.873298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:40.873298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:40.873298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:40.873298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:40.873298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:40.873298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:40.873298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:40.873298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:40.873298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:40.873298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:40.873298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:40.874327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:40.874327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:40.874327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:40.874327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:40.874327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:40.874327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:40.874327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:40.874327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:40.874327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:40.874327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:40.874327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:40.874327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:40.874327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:40.874327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:40.874327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:40.874327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:40.875273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:40.875273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:40.875273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:40.875273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:40.875273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:40.875273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:40.875273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:40.875273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:40.875273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:40.875273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:40.875273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:25:40.875273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:40.878546 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 356/688] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:25:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:40.881230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6667cc3-8a07-4120-9af4-11de8001c61f None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:40.882919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6667cc3-8a07-4120-9af4-11de8001c61f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:40.883353 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6667cc3-8a07-4120-9af4-11de8001c61f None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:40.885661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 357/689] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:40.887419 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6667cc3-8a07-4120-9af4-11de8001c61f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:40.914904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43dd98-7baf-4674-abd7-95e60cfde4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:40.914904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43dd98-7baf-4674-abd7-95e60cfde4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:40.914904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43dd98-7baf-4674-abd7-95e60cfde4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:40.914904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43dd98-7baf-4674-abd7-95e60cfde4b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:40.928657 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43dd98-7baf-4674-abd7-95e60cfde4b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:40.931129 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43dd98-7baf-4674-abd7-95e60cfde4b5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:40.940933 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:40.954844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-2c43dd98-7baf-4674-abd7-95e60cfde4b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:40.954844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:40.954844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:40.954844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:40.954844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:40.954844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:40.954844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:40.954844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:40.954844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:40.954844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:40.954844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:40.954844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:40.954844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:40.954844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:40.954844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:40.955803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:40.955803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:40.955803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:40.955803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:40.955803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:40.955803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:40.955803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:40.955803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:40.955803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:40.955803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:40.955803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:40.955803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:40.955803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:40.955803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:40.955803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:40.955803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:40.955803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:40.956709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:40.956709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:40.956709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:40.956709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:40.958821 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 333/690] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:40.960317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055afa3-dc57-4ade-897c-f88825800e8b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:40.960607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055afa3-dc57-4ade-897c-f88825800e8b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:40.960888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055afa3-dc57-4ade-897c-f88825800e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:40.980201 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2055afa3-dc57-4ade-897c-f88825800e8b None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:40.980920 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2055afa3-dc57-4ade-897c-f88825800e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:41.023816 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6667cc3-8a07-4120-9af4-11de8001c61f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:41.070943 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e712a-b092-4357-9d06-6aaa8ef4d294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:41.071647 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e712a-b092-4357-9d06-6aaa8ef4d294 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:41.071986 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 334/691] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:41.072108 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e712a-b092-4357-9d06-6aaa8ef4d294 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:41.072454 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e712a-b092-4357-9d06-6aaa8ef4d294 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:41.072833 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055afa3-dc57-4ade-897c-f88825800e8b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:41.073859 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e712a-b092-4357-9d06-6aaa8ef4d294 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:41.074236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e712a-b092-4357-9d06-6aaa8ef4d294 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:41.075769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e712a-b092-4357-9d06-6aaa8ef4d294 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:41.088419 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 358/692] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:25:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:41.094161 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e712a-b092-4357-9d06-6aaa8ef4d294 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:41.127371 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:41.209331 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e712a-b092-4357-9d06-6aaa8ef4d294 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:41.209830 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055afa3-dc57-4ade-897c-f88825800e8b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:41.240816 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:41.245768 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093359c-b04c-4ec6-9458-6cdd518aacd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:41.246866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093359c-b04c-4ec6-9458-6cdd518aacd0 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:41.247133 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093359c-b04c-4ec6-9458-6cdd518aacd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:41.247503 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093359c-b04c-4ec6-9458-6cdd518aacd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:41.250500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b85f6-9bd1-4989-b52a-4cf462bb3de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:41.251331 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b85f6-9bd1-4989-b52a-4cf462bb3de2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:41.251619 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b85f6-9bd1-4989-b52a-4cf462bb3de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:41.251894 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b85f6-9bd1-4989-b52a-4cf462bb3de2 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:41.252625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b85f6-9bd1-4989-b52a-4cf462bb3de2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:41.252932 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b85f6-9bd1-4989-b52a-4cf462bb3de2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:41.254346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b85f6-9bd1-4989-b52a-4cf462bb3de2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:41.259069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093359c-b04c-4ec6-9458-6cdd518aacd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:41.260524 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093359c-b04c-4ec6-9458-6cdd518aacd0 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:41.267737 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 335/693] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:41.268635 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b85f6-9bd1-4989-b52a-4cf462bb3de2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:41.289940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-5093359c-b04c-4ec6-9458-6cdd518aacd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:41.289940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:41.289940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:41.289940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:41.289940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:41.289940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:41.289940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:41.289940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:41.289940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:41.289940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:41.289940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:41.289940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:41.289940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:41.289940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:41.289940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:41.292105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:41.292105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:41.292105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:41.292105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:41.292105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:41.292105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:41.292105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:41.292105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:41.292105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:41.292105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:41.292105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:41.292105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:41.292105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:41.292105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:41.292105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:41.292105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:41.292105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:41.293398 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:41.293398 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:41.293398 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:41.293398 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:41.293398 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 359/694] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:25:41.293398 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093359c-b04c-4ec6-9458-6cdd518aacd0 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:41.384551 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:41.407281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b85f6-9bd1-4989-b52a-4cf462bb3de2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:41.408643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093359c-b04c-4ec6-9458-6cdd518aacd0 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:41.444905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f884218-4c5e-42af-8b8e-560f8812c7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:41.445861 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f884218-4c5e-42af-8b8e-560f8812c7fb None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:41.446246 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f884218-4c5e-42af-8b8e-560f8812c7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:41.446671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f884218-4c5e-42af-8b8e-560f8812c7fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:41.452911 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1fb92-2822-4c74-8129-e4fbcd7064f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:41.454144 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1fb92-2822-4c74-8129-e4fbcd7064f5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:41.454592 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1fb92-2822-4c74-8129-e4fbcd7064f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:41.454820 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f884218-4c5e-42af-8b8e-560f8812c7fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:41.455336 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1fb92-2822-4c74-8129-e4fbcd7064f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:41.456451 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f884218-4c5e-42af-8b8e-560f8812c7fb None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:41.467634 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1fb92-2822-4c74-8129-e4fbcd7064f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:41.469301 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1fb92-2822-4c74-8129-e4fbcd7064f5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:41.470919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-8f884218-4c5e-42af-8b8e-560f8812c7fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:41.470919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:41.470919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:41.470919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:41.470919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:41.470919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:41.470919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:41.470919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:41.470919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:41.470919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:41.470919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:41.470919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:41.470919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:25:41.470919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:25:41.470919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:25:41.471962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:25:41.471962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:41.471962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:41.471962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:41.471962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:41.471962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:41.471962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:41.471962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:41.471962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:41.471962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:41.471962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:41.471962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:41.471962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:41.471962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:41.471962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:41.471962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:41.471962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:41.473055 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:25:41.473055 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:25:41.473055 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:25:41.473055 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:25:41.473055 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:25:41.473055 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:25:41.473055 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:41.473055 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:41.479454 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 360/695] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:41.479454 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f884218-4c5e-42af-8b8e-560f8812c7fb None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:41.501992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f6f1fb92-2822-4c74-8129-e4fbcd7064f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:41.501992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:41.501992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:41.501992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:41.501992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:41.501992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:41.501992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:41.501992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:41.501992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:41.501992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:41.501992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:41.501992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:41.501992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:41.501992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:41.501992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:41.502967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:41.502967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:41.502967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:41.502967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:41.502967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:41.502967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:41.502967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:41.502967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:41.502967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:41.502967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:41.502967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:41.502967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:41.502967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:41.502967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:41.502967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:41.502967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:41.502967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:41.503868 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:41.503868 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:41.503868 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:41.503868 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:41.510917 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 336/696] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:41.512504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1fb92-2822-4c74-8129-e4fbcd7064f5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:41.569894 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:41.595122 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f884218-4c5e-42af-8b8e-560f8812c7fb None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:41.632146 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f40ca7-4eb6-484d-81bb-4ca715159f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:41.632810 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f40ca7-4eb6-484d-81bb-4ca715159f80 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:41.633096 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f40ca7-4eb6-484d-81bb-4ca715159f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:41.633372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f40ca7-4eb6-484d-81bb-4ca715159f80 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:41.634092 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f40ca7-4eb6-484d-81bb-4ca715159f80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:41.634401 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f40ca7-4eb6-484d-81bb-4ca715159f80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:41.635957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f40ca7-4eb6-484d-81bb-4ca715159f80 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:41.643108 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:41.650173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 361/697] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:41.651108 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f40ca7-4eb6-484d-81bb-4ca715159f80 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:41.653384 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1fb92-2822-4c74-8129-e4fbcd7064f5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:41.690394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a621c8-165e-414f-b22c-826942736cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:41.691129 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a621c8-165e-414f-b22c-826942736cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:41.691566 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a621c8-165e-414f-b22c-826942736cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:41.691672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a621c8-165e-414f-b22c-826942736cd4 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:41.692499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a621c8-165e-414f-b22c-826942736cd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:41.692751 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a621c8-165e-414f-b22c-826942736cd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:41.694229 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a621c8-165e-414f-b22c-826942736cd4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:41.711768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 337/698] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:25:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:41.711768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a621c8-165e-414f-b22c-826942736cd4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:41.773154 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f40ca7-4eb6-484d-81bb-4ca715159f80 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:41.809188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12a3bb-cf1a-4fbd-9163-cd85923a5ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:41.809983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12a3bb-cf1a-4fbd-9163-cd85923a5ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:41.810340 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12a3bb-cf1a-4fbd-9163-cd85923a5ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:41.810688 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12a3bb-cf1a-4fbd-9163-cd85923a5ec0 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:41.811738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12a3bb-cf1a-4fbd-9163-cd85923a5ec0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:41.811900 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12a3bb-cf1a-4fbd-9163-cd85923a5ec0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:41.813361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12a3bb-cf1a-4fbd-9163-cd85923a5ec0 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:41.829820 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 362/699] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:41.829820 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12a3bb-cf1a-4fbd-9163-cd85923a5ec0 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:41.849245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:41.853959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a621c8-165e-414f-b22c-826942736cd4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:41.898161 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:41.899343 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919f915-c8e9-4043-a4d2-82e5dff8e347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:41.900591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919f915-c8e9-4043-a4d2-82e5dff8e347 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:41.901021 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919f915-c8e9-4043-a4d2-82e5dff8e347 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:41.902023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919f915-c8e9-4043-a4d2-82e5dff8e347 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:41.911183 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7919f915-c8e9-4043-a4d2-82e5dff8e347 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:41.912843 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7919f915-c8e9-4043-a4d2-82e5dff8e347 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:41.931441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-7919f915-c8e9-4043-a4d2-82e5dff8e347 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 31 14:25:41.931441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:41.931441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:41.931441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:41.931441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:41.931441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:41.931441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:41.931441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:41.931441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:41.931441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:41.931441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:41.931441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:41.931441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:25:41.931441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:25:41.931441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:25:41.931441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:25:41.932422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:41.932422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:41.932422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:25:41.932422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:41.932422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:25:41.932422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:41.932422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:41.932422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:41.932422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:41.932422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:41.932422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:41.932422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:41.932422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:41.932422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:41.932422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:41.932422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:41.932422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:41.932422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:41.933385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:41.933385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:41.933385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:25:41.933385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:25:41.933385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:25:41.933385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:25:41.933385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:25:41.933385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:25:41.933385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 31 14:25:41.933385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:41.933910 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 338/700] 10.222.0.67 () {56 vars in 1093 bytes} [Tue May 31 14:25:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:41.934504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919f915-c8e9-4043-a4d2-82e5dff8e347 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:41.949509 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12a3bb-cf1a-4fbd-9163-cd85923a5ec0 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:41.998187 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4b1e7-6db2-4cc4-bcd0-af41f2ea092d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:41.999152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4b1e7-6db2-4cc4-bcd0-af41f2ea092d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:41.999435 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4b1e7-6db2-4cc4-bcd0-af41f2ea092d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:41.999708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4b1e7-6db2-4cc4-bcd0-af41f2ea092d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:42.001217 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4b1e7-6db2-4cc4-bcd0-af41f2ea092d None admin] RBAC: Adding query filter params (role.id=5956aa478bfc4d03919395e637e61c83, scope.project.id=f965a93a2f674598890bd8cafededb2c, user.id=dfa2282f957941b98db1d7497da3351b) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:42.001540 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4b1e7-6db2-4cc4-bcd0-af41f2ea092d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:42.003521 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4b1e7-6db2-4cc4-bcd0-af41f2ea092d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:42.014729 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 363/701] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:25:41 2022] GET /identity/v3/role_assignments?user.id=dfa2282f957941b98db1d7497da3351b&scope.project.id=f965a93a2f674598890bd8cafededb2c&role.id=5956aa478bfc4d03919395e637e61c83 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:42.015681 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4b1e7-6db2-4cc4-bcd0-af41f2ea092d None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:42.082816 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919f915-c8e9-4043-a4d2-82e5dff8e347 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:42.165096 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe6a29-ee96-4803-bb2e-13b69a25efbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:42.166037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe6a29-ee96-4803-bb2e-13b69a25efbd None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:42.166579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe6a29-ee96-4803-bb2e-13b69a25efbd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:42.167033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe6a29-ee96-4803-bb2e-13b69a25efbd None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:42.168536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe6a29-ee96-4803-bb2e-13b69a25efbd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:42.172853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe6a29-ee96-4803-bb2e-13b69a25efbd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:42.212292 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 339/702] 10.222.0.67 () {60 vars in 1132 bytes} [Tue May 31 14:25:41 2022] POST /identity/v3/users => generated 254 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:25:42.224062 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4b1e7-6db2-4cc4-bcd0-af41f2ea092d None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:42.250164 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:42.265362 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a30bf-e0df-46b4-ad16-2023ffefeaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:42.266301 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a30bf-e0df-46b4-ad16-2023ffefeaa2 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:42.266770 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a30bf-e0df-46b4-ad16-2023ffefeaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:42.267194 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a30bf-e0df-46b4-ad16-2023ffefeaa2 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:42.268235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a30bf-e0df-46b4-ad16-2023ffefeaa2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:42.268755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a30bf-e0df-46b4-ad16-2023ffefeaa2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:42.270486 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a30bf-e0df-46b4-ad16-2023ffefeaa2 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:42.296800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 364/703] 10.222.0.67 () {56 vars in 1133 bytes} [Tue May 31 14:25:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:42.302505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe6a29-ee96-4803-bb2e-13b69a25efbd None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:42.328100 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:42.428194 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe6a29-ee96-4803-bb2e-13b69a25efbd None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:42.467801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014576d9-276d-4f1f-aa1c-ecad4183f676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:42.468747 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014576d9-276d-4f1f-aa1c-ecad4183f676 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:42.469187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014576d9-276d-4f1f-aa1c-ecad4183f676 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:42.469643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014576d9-276d-4f1f-aa1c-ecad4183f676 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:42.547016 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014576d9-276d-4f1f-aa1c-ecad4183f676 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:42.548737 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014576d9-276d-4f1f-aa1c-ecad4183f676 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:42.566536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:42.662202 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 340/704] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:25:42.667498 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a30bf-e0df-46b4-ad16-2023ffefeaa2 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:42.784522 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a30bf-e0df-46b4-ad16-2023ffefeaa2 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:42.823091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67ceeb-c1b1-4e9e-ae3d-380b974681cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:42.824474 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67ceeb-c1b1-4e9e-ae3d-380b974681cb None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:42.824756 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67ceeb-c1b1-4e9e-ae3d-380b974681cb None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:42.825038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67ceeb-c1b1-4e9e-ae3d-380b974681cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:42.837341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67ceeb-c1b1-4e9e-ae3d-380b974681cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:42.838847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67ceeb-c1b1-4e9e-ae3d-380b974681cb None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:42.857173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-ba67ceeb-c1b1-4e9e-ae3d-380b974681cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:42.857173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:42.857173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:42.857173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:42.857173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:42.857173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:42.857173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:42.857173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:42.857173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:42.857173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:42.857173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:42.857173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:42.857173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:42.857173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:42.857173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:42.858079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:42.858079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:42.858079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:42.858079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:42.858079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:42.858079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:42.858079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:42.858079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:42.858079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:42.858079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:42.858079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:42.858079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:42.858079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:42.858079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:42.858079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:42.858079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:42.858079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:42.859010 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:42.859010 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:42.859010 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:42.859010 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:42.862769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 365/705] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:25:42.864166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014576d9-276d-4f1f-aa1c-ecad4183f676 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:43.007782 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014576d9-276d-4f1f-aa1c-ecad4183f676 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:43.024841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:43.044566 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb1de5-e9e6-4d28-ba8b-d79267ccda5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:43.045216 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb1de5-e9e6-4d28-ba8b-d79267ccda5c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:43.045502 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb1de5-e9e6-4d28-ba8b-d79267ccda5c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:43.045783 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb1de5-e9e6-4d28-ba8b-d79267ccda5c None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:43.046507 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb1de5-e9e6-4d28-ba8b-d79267ccda5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:43.046891 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb1de5-e9e6-4d28-ba8b-d79267ccda5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:43.048377 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb1de5-e9e6-4d28-ba8b-d79267ccda5c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:43.061085 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 341/706] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:43.067444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67ceeb-c1b1-4e9e-ae3d-380b974681cb None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:43.105724 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:43.178813 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67ceeb-c1b1-4e9e-ae3d-380b974681cb None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:43.214127 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94ac53-7bfd-4526-bb8b-d6cc2ab998b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:43.215019 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94ac53-7bfd-4526-bb8b-d6cc2ab998b1 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:43.215307 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94ac53-7bfd-4526-bb8b-d6cc2ab998b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:43.215582 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94ac53-7bfd-4526-bb8b-d6cc2ab998b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:43.221017 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990b814-8b15-4c10-8708-eeebf173d2d0 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:43.221492 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990b814-8b15-4c10-8708-eeebf173d2d0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:43.221571 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990b814-8b15-4c10-8708-eeebf173d2d0 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:43.223187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 342/707] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:43.226559 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc94ac53-7bfd-4526-bb8b-d6cc2ab998b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:43.228062 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc94ac53-7bfd-4526-bb8b-d6cc2ab998b1 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:43.233073 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4871d-9bc9-45e7-b429-a5f02981293a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:43.233564 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4871d-9bc9-45e7-b429-a5f02981293a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:43.233671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4871d-9bc9-45e7-b429-a5f02981293a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:43.246400 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-bc94ac53-7bfd-4526-bb8b-d6cc2ab998b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:43.246400 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:43.246400 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:43.246400 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:43.246400 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:43.246400 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:43.246400 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:43.246400 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:43.246400 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:43.246400 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:43.246400 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:43.246400 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:43.246400 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:25:43.246400 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:25:43.246400 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:25:43.247387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:25:43.247387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:43.247387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:43.247387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:43.247387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:43.247387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:43.247387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:43.247387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:43.247387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:43.247387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:43.247387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:43.247387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:43.247387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:43.247387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:43.247387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:43.247387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:43.247387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:43.248328 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:25:43.248328 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:25:43.248328 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:25:43.248328 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:25:43.248328 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:25:43.248328 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:25:43.248328 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:43.248328 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:43.250778 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 366/708] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:43.251748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-78d4871d-9bc9-45e7-b429-a5f02981293a None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:43.252928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2401d1-624e-4656-9caf-2c3ee16bd560 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:43.253068 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-78d4871d-9bc9-45e7-b429-a5f02981293a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:43.253609 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2401d1-624e-4656-9caf-2c3ee16bd560 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:43.254074 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2401d1-624e-4656-9caf-2c3ee16bd560 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:43.256003 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 367/709] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:43.256912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2401d1-624e-4656-9caf-2c3ee16bd560 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:43.333837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 343/710] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 31 14:25:43.335135 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842aa50d-71c1-45a7-a874-31896dbc4c12 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:43.335423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842aa50d-71c1-45a7-a874-31896dbc4c12 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:43.335699 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842aa50d-71c1-45a7-a874-31896dbc4c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:43.356137 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-842aa50d-71c1-45a7-a874-31896dbc4c12 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:43.357330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-842aa50d-71c1-45a7-a874-31896dbc4c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:43.385579 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:43.396841 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2401d1-624e-4656-9caf-2c3ee16bd560 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:43.433596 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82449880-a73f-4a2c-96e0-0c3fef260543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:43.434470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82449880-a73f-4a2c-96e0-0c3fef260543 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:43.434869 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82449880-a73f-4a2c-96e0-0c3fef260543 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:43.435237 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82449880-a73f-4a2c-96e0-0c3fef260543 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:43.436133 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82449880-a73f-4a2c-96e0-0c3fef260543 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:43.436535 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82449880-a73f-4a2c-96e0-0c3fef260543 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:43.438110 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82449880-a73f-4a2c-96e0-0c3fef260543 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:43.452360 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 344/711] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:43.453284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842aa50d-71c1-45a7-a874-31896dbc4c12 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:43.459313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 368/712] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:25:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:43.464483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82449880-a73f-4a2c-96e0-0c3fef260543 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:43.494950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:43.579019 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82449880-a73f-4a2c-96e0-0c3fef260543 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:43.583293 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842aa50d-71c1-45a7-a874-31896dbc4c12 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:43.614469 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa75a67-574c-4f98-b1d8-f41954568431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:43.615382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa75a67-574c-4f98-b1d8-f41954568431 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:43.615881 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa75a67-574c-4f98-b1d8-f41954568431 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:43.616299 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa75a67-574c-4f98-b1d8-f41954568431 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:43.617316 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa75a67-574c-4f98-b1d8-f41954568431 None admin] RBAC: Adding query filter params (role.id=e8d679596ffb45939229d1e2a13ce9b0, scope.project.id=f965a93a2f674598890bd8cafededb2c, user.id=68c6363b60954c3ebf50d2019f5f2fd2) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:43.617767 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa75a67-574c-4f98-b1d8-f41954568431 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:43.619362 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa75a67-574c-4f98-b1d8-f41954568431 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:43.624328 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f702714-4e01-4889-85b9-01cfb5623303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:43.625392 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f702714-4e01-4889-85b9-01cfb5623303 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:43.625684 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f702714-4e01-4889-85b9-01cfb5623303 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:43.625966 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f702714-4e01-4889-85b9-01cfb5623303 None admin] PATH_INFO: `/v3/services/network` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:43.626628 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f702714-4e01-4889-85b9-01cfb5623303 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:43.627317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 369/713] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:25:43 2022] GET /identity/v3/role_assignments?user.id=68c6363b60954c3ebf50d2019f5f2fd2&scope.project.id=f965a93a2f674598890bd8cafededb2c&role.id=e8d679596ffb45939229d1e2a13ce9b0 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:43.628223 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f702714-4e01-4889-85b9-01cfb5623303 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:43.628370 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa75a67-574c-4f98-b1d8-f41954568431 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:43.644759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-5f702714-4e01-4889-85b9-01cfb5623303 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 31 14:25:43.644759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:43.644759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:25:43.644759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:25:43.644759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:25:43.644759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:25:43.644759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:25:43.644759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:25:43.644759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 31 14:25:43.644759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:43.644759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:25:43.644759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:43.644759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:43.644759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:43.644759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:43.644759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:43.644759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:43.645999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:43.645999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:43.645999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:43.645999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:43.645999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:43.645999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:43.645999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:25:43.645999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:25:43.645999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:25:43.645999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:25:43.645999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:43.645999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:43.645999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:43.645999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:43.645999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:43.645999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:43.646984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:43.646984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:43.646984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:43.646984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:43.646984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:43.646984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:43.646984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:43.646984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:43.646984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:43.646984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:43.646984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:25:43.646984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:25:43.646984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 31 14:25:43.646984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:43.649432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 345/714] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:43 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:43.656798 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f702714-4e01-4889-85b9-01cfb5623303 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:43.668011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:43.779381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:43.783971 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f702714-4e01-4889-85b9-01cfb5623303 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:43.835895 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306b110-728c-437d-95f6-18efb62d4d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:43.836912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306b110-728c-437d-95f6-18efb62d4d50 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:43.837361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306b110-728c-437d-95f6-18efb62d4d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:43.837797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306b110-728c-437d-95f6-18efb62d4d50 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:43.838738 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1306b110-728c-437d-95f6-18efb62d4d50 None admin] RBAC: Adding query filter params (name=network) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:43.840183 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1306b110-728c-437d-95f6-18efb62d4d50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:43.843602 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1306b110-728c-437d-95f6-18efb62d4d50 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:43.854566 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 346/715] 10.222.0.67 () {56 vars in 1112 bytes} [Tue May 31 14:25:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:43.857143 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306b110-728c-437d-95f6-18efb62d4d50 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:43.888374 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa75a67-574c-4f98-b1d8-f41954568431 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:43.943986 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71226026-db5d-4734-8d48-7b1af0356439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:43.945279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71226026-db5d-4734-8d48-7b1af0356439 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:43.945773 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71226026-db5d-4734-8d48-7b1af0356439 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:43.946291 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71226026-db5d-4734-8d48-7b1af0356439 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:43.947196 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71226026-db5d-4734-8d48-7b1af0356439 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:43.949320 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71226026-db5d-4734-8d48-7b1af0356439 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:43.974672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306b110-728c-437d-95f6-18efb62d4d50 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:43.983188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-71226026-db5d-4734-8d48-7b1af0356439 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 31 14:25:43.983188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:43.983188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:25:43.983188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:25:43.983188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:25:43.983188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:25:43.983188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:25:43.983188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:25:43.983188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 31 14:25:43.983188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:43.983188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:25:43.983188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:43.983188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:43.983188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:43.983188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:43.983188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:43.984181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:43.984181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:43.984181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:43.984181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:43.984181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:43.984181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:43.984181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:43.984181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:25:43.984181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:25:43.984181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:25:43.984181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:25:43.984181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:43.984181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:43.984181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:43.984181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:43.984181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:43.985039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:43.985039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:43.985039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:43.985039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:43.985039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:43.985039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:43.985039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:43.985039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:43.985039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:43.985039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:43.985039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:43.985039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:25:43.985039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:25:43.985039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 31 14:25:43.985039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:43.988047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 370/716] 10.222.0.67 () {56 vars in 1129 bytes} [Tue May 31 14:25:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:43.993358 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71226026-db5d-4734-8d48-7b1af0356439 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:44.013353 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d479e7-eec0-48dd-974a-2a1961e3d1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:44.014048 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d479e7-eec0-48dd-974a-2a1961e3d1fc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:44.014338 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d479e7-eec0-48dd-974a-2a1961e3d1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:44.014948 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d479e7-eec0-48dd-974a-2a1961e3d1fc None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:44.015441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d479e7-eec0-48dd-974a-2a1961e3d1fc None admin] RBAC: Adding query filter params (type=network) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:44.015760 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d479e7-eec0-48dd-974a-2a1961e3d1fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:44.017139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d479e7-eec0-48dd-974a-2a1961e3d1fc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:44.027762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 347/717] 10.222.0.67 () {56 vars in 1112 bytes} [Tue May 31 14:25:43 2022] GET /identity/v3/services?type=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:44.159118 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:44.195085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71226026-db5d-4734-8d48-7b1af0356439 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:44.246610 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1e3dc-73a9-4945-bbb6-ae04df327d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:44.247303 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1e3dc-73a9-4945-bbb6-ae04df327d3b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:44.247584 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1e3dc-73a9-4945-bbb6-ae04df327d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:44.247855 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1e3dc-73a9-4945-bbb6-ae04df327d3b None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:44.248568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1e3dc-73a9-4945-bbb6-ae04df327d3b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:44.248876 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1e3dc-73a9-4945-bbb6-ae04df327d3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:44.250259 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1e3dc-73a9-4945-bbb6-ae04df327d3b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:44.256596 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 371/718] 10.222.0.67 () {56 vars in 1124 bytes} [Tue May 31 14:25:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:25:44.262697 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d479e7-eec0-48dd-974a-2a1961e3d1fc None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:44.306177 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:44.450057 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d479e7-eec0-48dd-974a-2a1961e3d1fc None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:44.512674 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84edc8bc-d858-47b6-ac3f-90fa78d95ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:44.513751 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edc8bc-d858-47b6-ac3f-90fa78d95ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:44.514229 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edc8bc-d858-47b6-ac3f-90fa78d95ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:44.514719 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edc8bc-d858-47b6-ac3f-90fa78d95ff4 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:44.516054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84edc8bc-d858-47b6-ac3f-90fa78d95ff4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:44.516602 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84edc8bc-d858-47b6-ac3f-90fa78d95ff4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:44.519003 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84edc8bc-d858-47b6-ac3f-90fa78d95ff4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:44.527956 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 348/719] 10.222.0.67 () {56 vars in 1124 bytes} [Tue May 31 14:25:44 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:45.181763 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d174f47-7ecb-4e53-a72a-cc4543766244 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:45.181763 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d174f47-7ecb-4e53-a72a-cc4543766244 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:45.181763 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d174f47-7ecb-4e53-a72a-cc4543766244 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:45.182550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 372/720] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:45.190046 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf594c-ffbb-4ab6-a4a4-6249f93063f4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:45.190046 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf594c-ffbb-4ab6-a4a4-6249f93063f4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:45.190046 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf594c-ffbb-4ab6-a4a4-6249f93063f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:45.197736 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d871296-e43b-43b6-bcbf-30fd454931dd None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:45.198324 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d871296-e43b-43b6-bcbf-30fd454931dd None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:45.198896 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d871296-e43b-43b6-bcbf-30fd454931dd None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:45.200721 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 373/721] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 31 14:25:45.205504 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532371d-a0df-47c7-b42f-94696f262bcb None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:45.206031 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532371d-a0df-47c7-b42f-94696f262bcb None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:45.206524 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532371d-a0df-47c7-b42f-94696f262bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:45.223660 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-92cf594c-ffbb-4ab6-a4a4-6249f93063f4 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:45.225203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-92cf594c-ffbb-4ab6-a4a4-6249f93063f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:45.236360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5532371d-a0df-47c7-b42f-94696f262bcb None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:45.242141 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5532371d-a0df-47c7-b42f-94696f262bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:45.348222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 374/722] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 31 14:25:45.373207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 349/723] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:45.381305 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532371d-a0df-47c7-b42f-94696f262bcb None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:45.413908 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf594c-ffbb-4ab6-a4a4-6249f93063f4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:45.451858 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:45.507341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:45.536615 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532371d-a0df-47c7-b42f-94696f262bcb None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:45.578215 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195bab2d-6c2c-4ef8-a8b4-60557945c0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:45.579463 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195bab2d-6c2c-4ef8-a8b4-60557945c0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:45.581321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195bab2d-6c2c-4ef8-a8b4-60557945c0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:45.581842 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195bab2d-6c2c-4ef8-a8b4-60557945c0e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:45.600644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf594c-ffbb-4ab6-a4a4-6249f93063f4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:45.605710 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195bab2d-6c2c-4ef8-a8b4-60557945c0e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:45.614403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-195bab2d-6c2c-4ef8-a8b4-60557945c0e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:45.614403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:45.614403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:45.614403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:45.614403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:45.614403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:45.614403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:45.614403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:45.614403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:45.614403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:45.614403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:45.614403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:45.614403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:45.614403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:45.614403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:45.614403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:45.615395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:45.615395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:45.615395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:45.615395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:45.615395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:45.615395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:45.615395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:45.615395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:45.615395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:45.615395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:45.615395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:45.615395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:45.615395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:45.615395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:45.615395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:45.615395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:45.616251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:45.616251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:45.616251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:45.616251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:45.616251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:45.616251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:45.616251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:45.616251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:45.616251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:45.616251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:45.616251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:45.616251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:45.627674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 375/724] 10.222.0.67 () {56 vars in 1102 bytes} [Tue May 31 14:25:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:45.636265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195bab2d-6c2c-4ef8-a8b4-60557945c0e3 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:45.679780 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920928b-a411-4c1c-83e7-e47f00ede7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:45.680601 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920928b-a411-4c1c-83e7-e47f00ede7bf None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:45.680889 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920928b-a411-4c1c-83e7-e47f00ede7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:45.681166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920928b-a411-4c1c-83e7-e47f00ede7bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:45.713726 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920928b-a411-4c1c-83e7-e47f00ede7bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:45.721512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-6920928b-a411-4c1c-83e7-e47f00ede7bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:25:45.721512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:45.721512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:45.721512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:45.721512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:45.721512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:45.721512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:45.721512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:45.721512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:45.721512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:45.721512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:45.721512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:45.721512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:45.721512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:45.721512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:45.721512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:45.722460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:45.722460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:45.722460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:45.722460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:45.722460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:45.722460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:45.722460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:45.722460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:45.722460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:45.722460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:45.722460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:45.722460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:45.722460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:45.722460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:45.722460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:45.722460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:45.723428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:45.723428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:45.723428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:45.723428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:45.723428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:45.723428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:45.723428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:45.723428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:45.723428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:45.723428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:45.723428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:25:45.723428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:45.729460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 350/725] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:25:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:45.730444 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920928b-a411-4c1c-83e7-e47f00ede7bf None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:45.755401 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195bab2d-6c2c-4ef8-a8b4-60557945c0e3 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:45.803161 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ac72f-0f5b-4ea3-9fc1-be2c440e986c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:45.803866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ac72f-0f5b-4ea3-9fc1-be2c440e986c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:45.804149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ac72f-0f5b-4ea3-9fc1-be2c440e986c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:45.804428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ac72f-0f5b-4ea3-9fc1-be2c440e986c None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:45.805219 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ac72f-0f5b-4ea3-9fc1-be2c440e986c None admin] RBAC: Adding query filter params (name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:45.805533 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ac72f-0f5b-4ea3-9fc1-be2c440e986c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:45.807008 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ac72f-0f5b-4ea3-9fc1-be2c440e986c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:45.809584 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:45.820552 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 376/726] 10.222.0.67 () {56 vars in 1103 bytes} [Tue May 31 14:25:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:45.829458 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ac72f-0f5b-4ea3-9fc1-be2c440e986c None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:45.944974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:45.966244 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920928b-a411-4c1c-83e7-e47f00ede7bf None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:45.976557 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ac72f-0f5b-4ea3-9fc1-be2c440e986c None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:46.001892 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8270c6-8aad-4483-aa25-a5e50533e808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:46.002570 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8270c6-8aad-4483-aa25-a5e50533e808 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:46.002885 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8270c6-8aad-4483-aa25-a5e50533e808 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:46.003167 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8270c6-8aad-4483-aa25-a5e50533e808 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:46.003921 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8270c6-8aad-4483-aa25-a5e50533e808 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:46.004232 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8270c6-8aad-4483-aa25-a5e50533e808 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:46.005613 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8270c6-8aad-4483-aa25-a5e50533e808 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:46.014493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 351/727] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:25:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:46.019967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8270c6-8aad-4483-aa25-a5e50533e808 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:46.068071 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9bce3-8ef0-4f22-af59-00298ca6d34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:46.068879 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9bce3-8ef0-4f22-af59-00298ca6d34d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:46.069162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9bce3-8ef0-4f22-af59-00298ca6d34d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:46.069435 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9bce3-8ef0-4f22-af59-00298ca6d34d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:46.092566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f9bce3-8ef0-4f22-af59-00298ca6d34d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:46.094025 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f9bce3-8ef0-4f22-af59-00298ca6d34d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:46.127810 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8270c6-8aad-4483-aa25-a5e50533e808 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:46.138958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-50f9bce3-8ef0-4f22-af59-00298ca6d34d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:46.138958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:46.138958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:46.138958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:46.138958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:46.138958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:46.138958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:46.138958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:46.138958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:46.138958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:46.138958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:46.138958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:46.138958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:46.138958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:46.138958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:46.139847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:46.139847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:46.139847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:46.139847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:46.139847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:46.139847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:46.139847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:46.139847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:46.139847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:46.139847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:46.139847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:46.139847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:46.139847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:46.139847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:46.139847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:46.139847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:46.139847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:46.140751 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:46.140751 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:46.140751 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:46.140751 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:46.140751 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 377/728] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:46.147443 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9bce3-8ef0-4f22-af59-00298ca6d34d None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:46.168640 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e9005-e74b-4b13-912e-471ec9b2466e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:46.169454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e9005-e74b-4b13-912e-471ec9b2466e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:46.169790 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e9005-e74b-4b13-912e-471ec9b2466e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:46.170076 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e9005-e74b-4b13-912e-471ec9b2466e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:46.181458 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55e9005-e74b-4b13-912e-471ec9b2466e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:46.183498 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55e9005-e74b-4b13-912e-471ec9b2466e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:46.186207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:46.203554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-c55e9005-e74b-4b13-912e-471ec9b2466e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:46.203554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:46.203554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:46.203554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:46.203554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:46.203554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:46.203554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:46.203554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:46.203554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:46.203554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:46.203554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:46.203554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:46.203554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:46.203554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:46.203554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:46.204480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:46.204480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:46.204480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:46.204480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:46.204480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:46.204480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:46.204480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:46.204480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:46.204480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:46.204480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:46.204480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:46.204480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:46.204480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:46.204480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:46.204480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:46.204480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:46.204480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:46.205381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:46.205381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:46.205381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:46.205381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:46.205381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 352/729] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:46.210431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e9005-e74b-4b13-912e-471ec9b2466e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:46.294728 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:46.318779 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e9005-e74b-4b13-912e-471ec9b2466e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:46.322763 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9bce3-8ef0-4f22-af59-00298ca6d34d None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:46.366791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee839a00-48a6-44b6-9fea-bb4e464634ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:46.367125 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88165af1-b2eb-4fd9-9f25-29761917298f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:46.367929 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88165af1-b2eb-4fd9-9f25-29761917298f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:46.368324 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88165af1-b2eb-4fd9-9f25-29761917298f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:46.368801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88165af1-b2eb-4fd9-9f25-29761917298f None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:46.369759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee839a00-48a6-44b6-9fea-bb4e464634ec None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:46.370585 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88165af1-b2eb-4fd9-9f25-29761917298f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:46.371145 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88165af1-b2eb-4fd9-9f25-29761917298f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:46.371493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee839a00-48a6-44b6-9fea-bb4e464634ec None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:46.372165 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee839a00-48a6-44b6-9fea-bb4e464634ec None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:46.373577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88165af1-b2eb-4fd9-9f25-29761917298f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:46.373942 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee839a00-48a6-44b6-9fea-bb4e464634ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:46.375170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee839a00-48a6-44b6-9fea-bb4e464634ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:46.377861 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee839a00-48a6-44b6-9fea-bb4e464634ec None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:46.406680 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 378/731] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:46.406680 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652a9ea-7c80-4220-b538-50d34eff4f81 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:46.406680 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652a9ea-7c80-4220-b538-50d34eff4f81 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:46.406680 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652a9ea-7c80-4220-b538-50d34eff4f81 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:46.407588 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 353/732] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:25:46.407742 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 379/732] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:46.408886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652a9ea-7c80-4220-b538-50d34eff4f81 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:46.416874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a7077-ea89-496b-a0f3-50144b31cd50 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:46.417656 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a7077-ea89-496b-a0f3-50144b31cd50 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:46.418360 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a7077-ea89-496b-a0f3-50144b31cd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:46.446145 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e58a7077-ea89-496b-a0f3-50144b31cd50 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:46.449150 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e58a7077-ea89-496b-a0f3-50144b31cd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:46.539827 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:46.576701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 354/733] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 31 14:25:46.578572 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a7077-ea89-496b-a0f3-50144b31cd50 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:46.588058 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652a9ea-7c80-4220-b538-50d34eff4f81 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:46.606626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:46.644058 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d8b65-beb9-4c7d-a320-72b333427a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:46.645406 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d8b65-beb9-4c7d-a320-72b333427a62 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:46.646151 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d8b65-beb9-4c7d-a320-72b333427a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:46.646721 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d8b65-beb9-4c7d-a320-72b333427a62 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:46.657476 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d8b65-beb9-4c7d-a320-72b333427a62 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:46.659957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d8b65-beb9-4c7d-a320-72b333427a62 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:46.679697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-f39d8b65-beb9-4c7d-a320-72b333427a62 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 31 14:25:46.679697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:46.679697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:46.679697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:46.679697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:46.679697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:46.679697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:46.679697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:46.679697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:46.679697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:46.679697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:46.679697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:46.679697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:25:46.679697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:25:46.679697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:25:46.679697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:25:46.680708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:46.680708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:46.680708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:25:46.680708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:46.680708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:25:46.680708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:46.680708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:46.680708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:46.680708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:46.680708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:46.680708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:46.680708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:46.680708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:46.680708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:46.680708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:46.680708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:46.680708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:46.680708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:46.681668 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:46.681668 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:46.681668 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:25:46.681668 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:25:46.681668 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:25:46.681668 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:25:46.681668 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:25:46.681668 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:25:46.681668 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 31 14:25:46.681668 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:46.688991 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 380/734] 10.222.0.67 () {56 vars in 1117 bytes} [Tue May 31 14:25:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:46.688991 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d8b65-beb9-4c7d-a320-72b333427a62 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:46.718418 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a7077-ea89-496b-a0f3-50144b31cd50 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:46.764147 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310df6e-c0fb-469b-a8c5-33b1a565fe06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:46.765344 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310df6e-c0fb-469b-a8c5-33b1a565fe06 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:46.766214 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310df6e-c0fb-469b-a8c5-33b1a565fe06 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:46.766714 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310df6e-c0fb-469b-a8c5-33b1a565fe06 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:46.776652 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310df6e-c0fb-469b-a8c5-33b1a565fe06 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:46.778404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310df6e-c0fb-469b-a8c5-33b1a565fe06 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:46.791314 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d8b65-beb9-4c7d-a320-72b333427a62 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:46.798156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e310df6e-c0fb-469b-a8c5-33b1a565fe06 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 31 14:25:46.798156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:46.798156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:46.798156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:46.798156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:46.798156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:46.798156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:46.798156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:46.798156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:46.798156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:46.798156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:46.798156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:46.798156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:25:46.798156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:25:46.798156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:25:46.798156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:25:46.799158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:46.799158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:46.799158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:25:46.799158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:46.799158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:25:46.799158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:46.799158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:46.799158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:46.799158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:46.799158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:46.799158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:46.799158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:46.799158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:46.799158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:46.799158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:46.799158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:46.799158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:46.799158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:46.800130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:46.800130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:46.800130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:25:46.800130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:25:46.800130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:25:46.800130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:25:46.800130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:25:46.800130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:25:46.800130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 31 14:25:46.800130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:46.807139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 355/735] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:25:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:46.807139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310df6e-c0fb-469b-a8c5-33b1a565fe06 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:46.837031 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0590e42-2824-4576-a668-16dbae0cb19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:46.837866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0590e42-2824-4576-a668-16dbae0cb19d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:46.838148 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0590e42-2824-4576-a668-16dbae0cb19d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:46.838420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0590e42-2824-4576-a668-16dbae0cb19d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:46.849536 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0590e42-2824-4576-a668-16dbae0cb19d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:46.858905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-e0590e42-2824-4576-a668-16dbae0cb19d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:25:46.858905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:46.858905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:46.858905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:46.858905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:46.858905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:46.858905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:46.858905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:46.858905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:46.858905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:46.858905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:46.858905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:46.858905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:46.858905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:46.858905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:46.858905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:46.859899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:46.859899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:46.859899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:46.859899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:46.859899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:46.859899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:46.859899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:46.859899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:46.859899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:46.859899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:46.859899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:46.859899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:46.859899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:46.859899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:46.859899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:46.859899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:46.860757 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:46.860757 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:46.860757 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:46.860757 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:46.860757 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:46.860757 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:46.860757 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:46.860757 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:46.860757 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:46.860757 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:46.860757 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:25:46.860757 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:46.861931 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 381/736] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:25:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:46.862548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0590e42-2824-4576-a668-16dbae0cb19d None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:46.871025 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:46.891870 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:46.933953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310df6e-c0fb-469b-a8c5-33b1a565fe06 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:46.969076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0590e42-2824-4576-a668-16dbae0cb19d None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:46.969552 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75546b5b-8683-4fe6-bcc4-7010d9ba78b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:46.970225 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75546b5b-8683-4fe6-bcc4-7010d9ba78b5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:46.970519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75546b5b-8683-4fe6-bcc4-7010d9ba78b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:46.970817 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75546b5b-8683-4fe6-bcc4-7010d9ba78b5 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:46.971660 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75546b5b-8683-4fe6-bcc4-7010d9ba78b5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:46.971969 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75546b5b-8683-4fe6-bcc4-7010d9ba78b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:46.973403 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75546b5b-8683-4fe6-bcc4-7010d9ba78b5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:46.992057 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 356/737] 10.222.0.67 () {56 vars in 1149 bytes} [Tue May 31 14:25:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:46.993342 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367680e-fd5b-4fc6-acd0-76a3792eb50c None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:46.993628 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367680e-fd5b-4fc6-acd0-76a3792eb50c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:46.993903 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367680e-fd5b-4fc6-acd0-76a3792eb50c None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:46.996006 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 357/738] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:46.996528 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367680e-fd5b-4fc6-acd0-76a3792eb50c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:47.014450 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02a90a-7d41-4dcb-b9c6-a0ed3ceab7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:47.015381 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02a90a-7d41-4dcb-b9c6-a0ed3ceab7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:47.015673 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02a90a-7d41-4dcb-b9c6-a0ed3ceab7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:47.015950 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02a90a-7d41-4dcb-b9c6-a0ed3ceab7a3 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:47.016790 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de02a90a-7d41-4dcb-b9c6-a0ed3ceab7a3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:47.017104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de02a90a-7d41-4dcb-b9c6-a0ed3ceab7a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:47.018555 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de02a90a-7d41-4dcb-b9c6-a0ed3ceab7a3 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:47.036480 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 382/739] 10.222.0.67 () {56 vars in 1135 bytes} [Tue May 31 14:25:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:47.037782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92844c98-d4aa-49a1-90cb-72a007b46fe5 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:47.038073 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92844c98-d4aa-49a1-90cb-72a007b46fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:47.038351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92844c98-d4aa-49a1-90cb-72a007b46fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:47.060169 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-92844c98-d4aa-49a1-90cb-72a007b46fe5 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:47.061750 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-92844c98-d4aa-49a1-90cb-72a007b46fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:47.111595 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367680e-fd5b-4fc6-acd0-76a3792eb50c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:47.115804 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:47.142382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 383/740] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:47.143266 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92844c98-d4aa-49a1-90cb-72a007b46fe5 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:47.158951 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a59d97-1fa0-452f-b7e9-9fc2c97442b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:47.159626 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a59d97-1fa0-452f-b7e9-9fc2c97442b1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:47.159915 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a59d97-1fa0-452f-b7e9-9fc2c97442b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:47.160198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a59d97-1fa0-452f-b7e9-9fc2c97442b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:47.160938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a59d97-1fa0-452f-b7e9-9fc2c97442b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:47.161246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a59d97-1fa0-452f-b7e9-9fc2c97442b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:47.162618 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a59d97-1fa0-452f-b7e9-9fc2c97442b1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:47.172271 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 358/741] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:25:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:47.173085 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a59d97-1fa0-452f-b7e9-9fc2c97442b1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:47.179695 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:47.265907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92844c98-d4aa-49a1-90cb-72a007b46fe5 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:47.278767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a59d97-1fa0-452f-b7e9-9fc2c97442b1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:47.301213 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609ef04-df5d-45a0-a752-5a459497f168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:47.301862 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609ef04-df5d-45a0-a752-5a459497f168 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:47.302145 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609ef04-df5d-45a0-a752-5a459497f168 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:47.302422 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609ef04-df5d-45a0-a752-5a459497f168 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:47.314007 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6a6c8-4c9e-4764-898c-bd3ad7a48610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:47.314603 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6a6c8-4c9e-4764-898c-bd3ad7a48610 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:47.314919 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6a6c8-4c9e-4764-898c-bd3ad7a48610 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:47.315194 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6a6c8-4c9e-4764-898c-bd3ad7a48610 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:47.350070 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:47.373605 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609ef04-df5d-45a0-a752-5a459497f168 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:47.375107 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609ef04-df5d-45a0-a752-5a459497f168 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:47.392894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:47.393987 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d6a6c8-4c9e-4764-898c-bd3ad7a48610 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:47.395741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d6a6c8-4c9e-4764-898c-bd3ad7a48610 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:47.476988 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 384/742] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:25:47.477800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609ef04-df5d-45a0-a752-5a459497f168 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:47.500587 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 359/743] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 31 14:25:47.500587 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6a6c8-4c9e-4764-898c-bd3ad7a48610 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:47.560082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:47.576987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609ef04-df5d-45a0-a752-5a459497f168 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:47.593409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:47.594741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6a6c8-4c9e-4764-898c-bd3ad7a48610 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:47.611838 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3b622-6f34-45db-8bf1-aef624594cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:47.612490 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3b622-6f34-45db-8bf1-aef624594cc4 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:47.612771 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3b622-6f34-45db-8bf1-aef624594cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:47.613046 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3b622-6f34-45db-8bf1-aef624594cc4 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:47.613762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a3b622-6f34-45db-8bf1-aef624594cc4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:47.615181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a3b622-6f34-45db-8bf1-aef624594cc4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:47.628721 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554be37-3437-477a-bf77-d077f6e842b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:47.629048 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 385/744] 10.222.0.67 () {60 vars in 1141 bytes} [Tue May 31 14:25:47 2022] POST /identity/v3/services => generated 240 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:25:47.629522 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554be37-3437-477a-bf77-d077f6e842b5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:47.629812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554be37-3437-477a-bf77-d077f6e842b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:47.630296 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9247842-9917-480b-8719-e9813869079d None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:47.630459 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554be37-3437-477a-bf77-d077f6e842b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:47.630611 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9247842-9917-480b-8719-e9813869079d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:47.630881 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9247842-9917-480b-8719-e9813869079d None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:47.634701 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 386/745] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:47.634701 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9247842-9917-480b-8719-e9813869079d None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:47.648953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554be37-3437-477a-bf77-d077f6e842b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:47.650989 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554be37-3437-477a-bf77-d077f6e842b5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:47.672630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-8554be37-3437-477a-bf77-d077f6e842b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:47.672630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:47.672630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:47.672630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:47.672630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:47.672630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:47.672630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:47.672630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:47.672630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:47.672630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:47.672630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:47.672630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:47.672630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:47.672630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:47.672630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:47.673778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:47.673778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:47.673778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:47.673778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:47.673778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:47.673778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:47.673778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:47.673778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:47.673778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:47.673778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:47.673778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:47.673778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:47.673778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:47.673778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:47.673778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:47.673778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:47.673778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:47.674892 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:47.674892 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:47.674892 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:47.674892 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:47.676055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 360/746] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:47.676910 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554be37-3437-477a-bf77-d077f6e842b5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:47.765173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9247842-9917-480b-8719-e9813869079d None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:47.792378 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554be37-3437-477a-bf77-d077f6e842b5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:47.801429 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc748ce-2eff-44af-8889-be6de47209d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:47.802246 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc748ce-2eff-44af-8889-be6de47209d9 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:47.802528 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc748ce-2eff-44af-8889-be6de47209d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:47.802845 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc748ce-2eff-44af-8889-be6de47209d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:47.813659 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:47.814935 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc748ce-2eff-44af-8889-be6de47209d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:47.816546 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc748ce-2eff-44af-8889-be6de47209d9 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:47.835774 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a1a57-97f4-4f90-a491-dc06d2ac8aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:47.836509 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a1a57-97f4-4f90-a491-dc06d2ac8aeb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:47.836796 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a1a57-97f4-4f90-a491-dc06d2ac8aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:47.836961 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-3bc748ce-2eff-44af-8889-be6de47209d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:47.836961 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:47.836961 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:47.836961 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:47.836961 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:47.836961 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:47.836961 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:47.836961 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:47.836961 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:47.836961 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:47.836961 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:47.836961 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:47.836961 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:47.836961 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:47.836961 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:47.837980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:47.837980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:47.837980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:47.837980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:47.837980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:47.837980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:47.837980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:47.837980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:47.837980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:47.837980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:47.837980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:47.837980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:47.837980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:47.837980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:47.837980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:47.837980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:47.837980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:47.839626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:47.839626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:47.839626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:47.839626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:47.839849 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a1a57-97f4-4f90-a491-dc06d2ac8aeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:47.839919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 387/747] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:47.840413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f727d2d-7b46-4ecb-bfe2-0bc42f3eee03 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:47.840704 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f727d2d-7b46-4ecb-bfe2-0bc42f3eee03 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:47.840985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f727d2d-7b46-4ecb-bfe2-0bc42f3eee03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:47.857529 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a1a57-97f4-4f90-a491-dc06d2ac8aeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:47.859022 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a1a57-97f4-4f90-a491-dc06d2ac8aeb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:47.859550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-0f727d2d-7b46-4ecb-bfe2-0bc42f3eee03 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:47.860774 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0f727d2d-7b46-4ecb-bfe2-0bc42f3eee03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:47.879422 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:47.880322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-1b2a1a57-97f4-4f90-a491-dc06d2ac8aeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:47.880322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:47.880322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:47.880322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:47.880322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:47.880322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:47.880322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:47.880322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:47.880322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:47.880322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:47.880322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:47.880322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:47.880322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:47.880322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:47.880322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:47.881313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:47.881313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:47.881313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:47.881313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:47.881313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:47.881313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:47.881313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:47.881313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:47.881313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:47.881313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:47.881313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:47.881313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:47.881313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:47.881313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:47.881313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:47.881313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:47.881313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:47.882228 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:47.882228 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:47.882228 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:47.882228 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:47.882228 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 361/748] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:47.883535 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a1a57-97f4-4f90-a491-dc06d2ac8aeb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:47.938570 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 388/749] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:47 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 31 14:25:47.939461 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f727d2d-7b46-4ecb-bfe2-0bc42f3eee03 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:48.000368 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a1a57-97f4-4f90-a491-dc06d2ac8aeb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:48.037794 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef873914-6158-4643-b56f-ae016a67286c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:48.038469 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef873914-6158-4643-b56f-ae016a67286c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:48.038789 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef873914-6158-4643-b56f-ae016a67286c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:48.039069 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef873914-6158-4643-b56f-ae016a67286c None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:48.039790 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef873914-6158-4643-b56f-ae016a67286c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:48.040105 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef873914-6158-4643-b56f-ae016a67286c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:48.041470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef873914-6158-4643-b56f-ae016a67286c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:48.042378 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f727d2d-7b46-4ecb-bfe2-0bc42f3eee03 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:48.058695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 362/750] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:48.058695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef873914-6158-4643-b56f-ae016a67286c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:48.078332 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f822ade-0998-4010-b870-6a6cedbde20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:48.079025 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f822ade-0998-4010-b870-6a6cedbde20b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:48.079316 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f822ade-0998-4010-b870-6a6cedbde20b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:48.079590 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f822ade-0998-4010-b870-6a6cedbde20b None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:48.080312 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f822ade-0998-4010-b870-6a6cedbde20b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:48.080618 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f822ade-0998-4010-b870-6a6cedbde20b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:48.082001 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f822ade-0998-4010-b870-6a6cedbde20b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:48.092116 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:48.097698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 389/751] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:48.097698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f822ade-0998-4010-b870-6a6cedbde20b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:48.112253 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:48.179968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef873914-6158-4643-b56f-ae016a67286c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:48.202890 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f822ade-0998-4010-b870-6a6cedbde20b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:48.215184 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d4a57-0d94-4086-8dff-7ff27135420e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:48.215856 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d4a57-0d94-4086-8dff-7ff27135420e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:48.216143 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d4a57-0d94-4086-8dff-7ff27135420e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:48.216434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d4a57-0d94-4086-8dff-7ff27135420e None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:48.217196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d4a57-0d94-4086-8dff-7ff27135420e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:48.217512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d4a57-0d94-4086-8dff-7ff27135420e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:48.218904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d4a57-0d94-4086-8dff-7ff27135420e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:48.235356 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 363/752] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:48.235356 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d4a57-0d94-4086-8dff-7ff27135420e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:48.243354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc23edc-ba10-4c05-9d0b-1f821d3ff78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:48.244093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc23edc-ba10-4c05-9d0b-1f821d3ff78d None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:48.244378 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc23edc-ba10-4c05-9d0b-1f821d3ff78d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:48.244688 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc23edc-ba10-4c05-9d0b-1f821d3ff78d None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:48.246025 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc23edc-ba10-4c05-9d0b-1f821d3ff78d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:48.247439 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc23edc-ba10-4c05-9d0b-1f821d3ff78d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:48.261809 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 390/753] 10.222.0.67 () {60 vars in 1141 bytes} [Tue May 31 14:25:48 2022] POST /identity/v3/services => generated 251 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:25:48.262976 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc23edc-ba10-4c05-9d0b-1f821d3ff78d None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:48.361525 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:48.377910 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:48.393378 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d4a57-0d94-4086-8dff-7ff27135420e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:48.411373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc23edc-ba10-4c05-9d0b-1f821d3ff78d None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:48.434687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981e3cd-32df-435f-bb39-d9a74ff7b939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:48.435678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981e3cd-32df-435f-bb39-d9a74ff7b939 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:48.436069 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981e3cd-32df-435f-bb39-d9a74ff7b939 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:48.436454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981e3cd-32df-435f-bb39-d9a74ff7b939 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:48.445212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b981e3cd-32df-435f-bb39-d9a74ff7b939 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:48.447139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b981e3cd-32df-435f-bb39-d9a74ff7b939 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:48.460850 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab945d2-1712-4b90-a1bd-c68b32bcdae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:48.461932 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab945d2-1712-4b90-a1bd-c68b32bcdae8 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:48.462375 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab945d2-1712-4b90-a1bd-c68b32bcdae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:48.462673 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab945d2-1712-4b90-a1bd-c68b32bcdae8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:48.466027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-b981e3cd-32df-435f-bb39-d9a74ff7b939 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 31 14:25:48.466027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:48.466027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:48.466027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:48.466027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:48.466027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:48.466027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:48.466027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:48.466027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:48.466027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:48.466027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:48.466027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:48.466027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:25:48.466027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:25:48.466027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:25:48.466027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:25:48.467011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:48.467011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:48.467011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:25:48.467011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:48.467011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:25:48.467011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:48.467011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:48.467011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:48.467011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:48.467011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:48.467011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:48.467011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:48.467011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:48.467011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:48.467011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:48.467011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:48.467011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:48.467011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:48.467959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:48.467959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:48.467959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:25:48.467959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:25:48.467959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:25:48.467959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:25:48.467959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:25:48.467959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:25:48.467959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 31 14:25:48.467959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:48.470253 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab945d2-1712-4b90-a1bd-c68b32bcdae8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:48.471025 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 364/754] 10.222.0.67 () {56 vars in 1093 bytes} [Tue May 31 14:25:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:48.471876 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab945d2-1712-4b90-a1bd-c68b32bcdae8 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:48.472329 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981e3cd-32df-435f-bb39-d9a74ff7b939 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:48.491449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-4ab945d2-1712-4b90-a1bd-c68b32bcdae8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:48.491449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:48.491449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:48.491449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:48.491449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:48.491449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:48.491449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:48.491449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:48.491449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:48.491449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:48.491449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:48.491449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:48.491449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:25:48.491449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:25:48.491449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:25:48.492417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:25:48.492417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:48.492417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:48.492417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:48.492417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:48.492417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:48.492417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:48.492417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:48.492417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:48.492417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:48.492417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:48.492417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:48.492417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:48.492417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:48.492417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:48.492417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:48.492417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:48.493311 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:25:48.493311 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:25:48.493311 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:25:48.493311 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:25:48.493311 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:25:48.493311 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:25:48.493311 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:48.493311 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:48.493760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 391/755] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:48.494283 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab945d2-1712-4b90-a1bd-c68b32bcdae8 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:48.609699 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981e3cd-32df-435f-bb39-d9a74ff7b939 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:48.627549 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab945d2-1712-4b90-a1bd-c68b32bcdae8 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:48.647507 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624229a1-6038-4116-be71-5941e1a0a720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:48.648445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624229a1-6038-4116-be71-5941e1a0a720 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:48.648832 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624229a1-6038-4116-be71-5941e1a0a720 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:48.649205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624229a1-6038-4116-be71-5941e1a0a720 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:48.656305 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624229a1-6038-4116-be71-5941e1a0a720 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:48.657876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624229a1-6038-4116-be71-5941e1a0a720 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:48.661023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:48.665049 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44aa44-156d-40a6-9105-6987435d4f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:48.665749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44aa44-156d-40a6-9105-6987435d4f9a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:48.666081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44aa44-156d-40a6-9105-6987435d4f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:48.666408 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44aa44-156d-40a6-9105-6987435d4f9a None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:48.668351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c44aa44-156d-40a6-9105-6987435d4f9a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:48.668729 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c44aa44-156d-40a6-9105-6987435d4f9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:48.670165 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c44aa44-156d-40a6-9105-6987435d4f9a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:48.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-624229a1-6038-4116-be71-5941e1a0a720 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:48.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:48.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:48.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:48.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:48.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:48.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:48.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:48.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:48.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:48.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:48.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:48.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:25:48.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:25:48.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:25:48.673680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:25:48.673680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:48.673680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:48.673680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:48.673680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:48.673680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:48.673680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:48.673680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:48.673680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:48.673680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:48.673680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:48.673680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:48.673680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:48.673680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:48.673680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:48.673680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:48.673680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:48.674606 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:25:48.674606 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:25:48.674606 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:25:48.674606 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:25:48.674606 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:25:48.674606 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:25:48.674606 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:48.674606 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:48.674606 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 365/756] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:48.675172 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624229a1-6038-4116-be71-5941e1a0a720 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:48.689228 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:48.701230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 392/757] 10.222.0.67 () {56 vars in 1133 bytes} [Tue May 31 14:25:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:48.703076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44aa44-156d-40a6-9105-6987435d4f9a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:48.783136 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624229a1-6038-4116-be71-5941e1a0a720 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:48.815569 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44aa44-156d-40a6-9105-6987435d4f9a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:48.818824 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f19ae0-b68b-47c4-b8ab-f651bd6a59cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:48.819182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f19ae0-b68b-47c4-b8ab-f651bd6a59cf None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:48.819524 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f19ae0-b68b-47c4-b8ab-f651bd6a59cf None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:48.819803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f19ae0-b68b-47c4-b8ab-f651bd6a59cf None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:48.820644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f19ae0-b68b-47c4-b8ab-f651bd6a59cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:48.820960 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f19ae0-b68b-47c4-b8ab-f651bd6a59cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:48.822395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f19ae0-b68b-47c4-b8ab-f651bd6a59cf None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:48.840741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 366/758] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:25:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:48.840741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f19ae0-b68b-47c4-b8ab-f651bd6a59cf None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:48.861260 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9520f-7dee-4e83-805d-47e02deba6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:48.862275 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9520f-7dee-4e83-805d-47e02deba6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:48.862746 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9520f-7dee-4e83-805d-47e02deba6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:48.863294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9520f-7dee-4e83-805d-47e02deba6c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:48.864232 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c9520f-7dee-4e83-805d-47e02deba6c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:48.864691 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c9520f-7dee-4e83-805d-47e02deba6c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:48.866504 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c9520f-7dee-4e83-805d-47e02deba6c4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:48.880121 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 393/759] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:25:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:48.881331 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9520f-7dee-4e83-805d-47e02deba6c4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:48.911062 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:48.946405 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f19ae0-b68b-47c4-b8ab-f651bd6a59cf None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:48.948626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:48.990425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23813f4a-bcb4-4d8a-aadf-2a6911d58d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:48.992219 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23813f4a-bcb4-4d8a-aadf-2a6911d58d73 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:48.992742 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23813f4a-bcb4-4d8a-aadf-2a6911d58d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:48.993224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23813f4a-bcb4-4d8a-aadf-2a6911d58d73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:49.007482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9520f-7dee-4e83-805d-47e02deba6c4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:49.008299 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23813f4a-bcb4-4d8a-aadf-2a6911d58d73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:49.011247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23813f4a-bcb4-4d8a-aadf-2a6911d58d73 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:49.034886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-23813f4a-bcb4-4d8a-aadf-2a6911d58d73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:49.034886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:49.034886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:49.034886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:49.034886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:49.034886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:49.034886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:49.034886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:49.034886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:49.034886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:49.034886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:49.034886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:49.034886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:49.034886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:49.034886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:49.035823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:49.035823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:49.035823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:49.035823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:49.035823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:49.035823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:49.035823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:49.035823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:49.035823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:49.035823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:49.035823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:49.035823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:49.035823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:49.035823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:49.035823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:49.035823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:49.035823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:49.036740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:49.036740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:49.036740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:49.036740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:49.044620 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 367/760] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:49.044620 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23813f4a-bcb4-4d8a-aadf-2a6911d58d73 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:49.058477 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab35344-1393-4834-aed9-49996723b17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:49.059760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab35344-1393-4834-aed9-49996723b17b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:49.060247 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab35344-1393-4834-aed9-49996723b17b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:49.061003 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab35344-1393-4834-aed9-49996723b17b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:49.062526 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab35344-1393-4834-aed9-49996723b17b None admin] RBAC: Adding query filter params (role.id=5956aa478bfc4d03919395e637e61c83, scope.project.id=f965a93a2f674598890bd8cafededb2c, user.id=0b7d100f2407432381d119c25814fff3) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:49.063074 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab35344-1393-4834-aed9-49996723b17b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:49.064816 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab35344-1393-4834-aed9-49996723b17b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:49.079477 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 394/761] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:25:48 2022] GET /identity/v3/role_assignments?user.id=0b7d100f2407432381d119c25814fff3&scope.project.id=f965a93a2f674598890bd8cafededb2c&role.id=5956aa478bfc4d03919395e637e61c83 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:49.080700 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab35344-1393-4834-aed9-49996723b17b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:49.165403 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23813f4a-bcb4-4d8a-aadf-2a6911d58d73 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:49.189951 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:49.203164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907ea86-9bb9-493d-8bd9-ec7fa0c00fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:49.204265 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907ea86-9bb9-493d-8bd9-ec7fa0c00fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:49.204987 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907ea86-9bb9-493d-8bd9-ec7fa0c00fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:49.205365 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907ea86-9bb9-493d-8bd9-ec7fa0c00fb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:49.206371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907ea86-9bb9-493d-8bd9-ec7fa0c00fb8 None admin] RBAC: Adding query filter params (role.id=e8d679596ffb45939229d1e2a13ce9b0, scope.project.id=f965a93a2f674598890bd8cafededb2c, user.id=dfa2282f957941b98db1d7497da3351b) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:49.206794 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907ea86-9bb9-493d-8bd9-ec7fa0c00fb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:49.208376 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907ea86-9bb9-493d-8bd9-ec7fa0c00fb8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:49.214400 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab35344-1393-4834-aed9-49996723b17b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:49.215414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 368/762] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:25:49 2022] GET /identity/v3/role_assignments?user.id=dfa2282f957941b98db1d7497da3351b&scope.project.id=f965a93a2f674598890bd8cafededb2c&role.id=e8d679596ffb45939229d1e2a13ce9b0 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:49.231259 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:49.259137 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2f95a-4c5c-4c0c-b83c-7ba13a252734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:49.260094 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2f95a-4c5c-4c0c-b83c-7ba13a252734 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:49.261081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2f95a-4c5c-4c0c-b83c-7ba13a252734 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:49.261572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2f95a-4c5c-4c0c-b83c-7ba13a252734 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:49.262507 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2f95a-4c5c-4c0c-b83c-7ba13a252734 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:49.263017 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2f95a-4c5c-4c0c-b83c-7ba13a252734 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:49.264630 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2f95a-4c5c-4c0c-b83c-7ba13a252734 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:49.279732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 395/763] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:49.285523 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907ea86-9bb9-493d-8bd9-ec7fa0c00fb8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:49.404425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907ea86-9bb9-493d-8bd9-ec7fa0c00fb8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:49.441582 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6225d5a2-5454-4565-b994-0425a345bbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:49.442396 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225d5a2-5454-4565-b994-0425a345bbf9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:49.442765 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225d5a2-5454-4565-b994-0425a345bbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:49.443052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225d5a2-5454-4565-b994-0425a345bbf9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:49.451786 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6225d5a2-5454-4565-b994-0425a345bbf9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:49.453340 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6225d5a2-5454-4565-b994-0425a345bbf9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:49.469288 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-6225d5a2-5454-4565-b994-0425a345bbf9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:49.469288 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:49.469288 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:49.469288 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:49.469288 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:49.469288 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:49.469288 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:49.469288 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:49.469288 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:49.469288 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:49.469288 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:49.469288 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:49.469288 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:25:49.469288 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:25:49.469288 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:25:49.470268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:25:49.470268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:49.470268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:49.470268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:49.470268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:49.470268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:49.470268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:49.470268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:49.470268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:49.470268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:49.470268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:49.470268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:49.470268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:49.470268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:49.470268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:49.470268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:49.470268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:49.471188 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:25:49.471188 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:25:49.471188 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:25:49.471188 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:25:49.471188 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:25:49.471188 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:25:49.471188 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:49.471188 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:49.471188 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 369/764] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:49.478543 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2f95a-4c5c-4c0c-b83c-7ba13a252734 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:49.601574 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2f95a-4c5c-4c0c-b83c-7ba13a252734 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:49.647157 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9de3e7-a2eb-4624-a37c-64575a250f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:49.647833 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9de3e7-a2eb-4624-a37c-64575a250f1f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:49.648112 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9de3e7-a2eb-4624-a37c-64575a250f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:49.648389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9de3e7-a2eb-4624-a37c-64575a250f1f None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:49.649185 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9de3e7-a2eb-4624-a37c-64575a250f1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:49.649495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9de3e7-a2eb-4624-a37c-64575a250f1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:49.651023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9de3e7-a2eb-4624-a37c-64575a250f1f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:49.667894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 396/765] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:25:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:25:49.673995 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6225d5a2-5454-4565-b994-0425a345bbf9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:49.740962 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:49.820513 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6225d5a2-5454-4565-b994-0425a345bbf9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:49.873761 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8718aa-be74-48f6-8429-3c0e714ba23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:49.874696 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8718aa-be74-48f6-8429-3c0e714ba23b None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:49.874991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8718aa-be74-48f6-8429-3c0e714ba23b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:49.875270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8718aa-be74-48f6-8429-3c0e714ba23b None admin] PATH_INFO: `/v3/projects/f965a93a2f674598890bd8cafededb2c/users/68c6363b60954c3ebf50d2019f5f2fd2/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:49.877284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8718aa-be74-48f6-8429-3c0e714ba23b None admin] RBAC: Authorizing `identity:create_grant(project_id=f965a93a2f674598890bd8cafededb2c, user_id=68c6363b60954c3ebf50d2019f5f2fd2, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:49.878753 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8718aa-be74-48f6-8429-3c0e714ba23b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:49.897681 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 370/766] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:25:49 2022] PUT /identity/v3/projects/f965a93a2f674598890bd8cafededb2c/users/68c6363b60954c3ebf50d2019f5f2fd2/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:25:50.209616 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67390a5e-b295-4cac-ba20-4251dad00868 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:50.209962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67390a5e-b295-4cac-ba20-4251dad00868 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:50.210191 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67390a5e-b295-4cac-ba20-4251dad00868 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:50.211768 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 397/767] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:50.217820 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36814953-6c25-48a4-974b-a7109340d2b4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:50.218117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36814953-6c25-48a4-974b-a7109340d2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:50.218401 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36814953-6c25-48a4-974b-a7109340d2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:50.237706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-36814953-6c25-48a4-974b-a7109340d2b4 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:50.239001 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-36814953-6c25-48a4-974b-a7109340d2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:50.335294 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 371/768] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:25:50.380165 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67390a5e-b295-4cac-ba20-4251dad00868 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:50.426069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:50.517579 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67390a5e-b295-4cac-ba20-4251dad00868 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:50.552317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14c427-f076-4b3b-b9cf-a7eb6da4d11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:50.553150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14c427-f076-4b3b-b9cf-a7eb6da4d11c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:50.553435 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14c427-f076-4b3b-b9cf-a7eb6da4d11c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:50.553714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14c427-f076-4b3b-b9cf-a7eb6da4d11c None admin] PATH_INFO: `/v3/services/network` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:50.554395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14c427-f076-4b3b-b9cf-a7eb6da4d11c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:50.556055 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14c427-f076-4b3b-b9cf-a7eb6da4d11c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:50.569095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-4a14c427-f076-4b3b-b9cf-a7eb6da4d11c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 31 14:25:50.569095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:50.569095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:25:50.569095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:25:50.569095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:25:50.569095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:25:50.569095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:25:50.569095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:25:50.569095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 31 14:25:50.569095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:50.569095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:25:50.569095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:50.569095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:50.569095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:50.569095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:50.569095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:50.569095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:50.570149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:50.570149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:50.570149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:50.570149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:50.570149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:50.570149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:50.570149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:25:50.570149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:25:50.570149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:25:50.570149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:25:50.570149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:50.570149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:50.570149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:50.570149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:50.570149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:50.570149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:50.571050 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:50.571050 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:50.571050 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:50.571050 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:50.571050 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:50.571050 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:50.571050 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:50.571050 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:50.571050 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:50.571050 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:50.571050 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:25:50.571050 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:25:50.571050 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 31 14:25:50.571050 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:50.574037 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 398/769] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:50 2022] GET /identity/v3/services/network => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:50.577640 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36814953-6c25-48a4-974b-a7109340d2b4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:50.623770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:50.697395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36814953-6c25-48a4-974b-a7109340d2b4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:50.734872 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29f42f-6f37-4586-bd2d-722d74a05159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:50.735554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29f42f-6f37-4586-bd2d-722d74a05159 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:50.735841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29f42f-6f37-4586-bd2d-722d74a05159 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:50.736116 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29f42f-6f37-4586-bd2d-722d74a05159 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:50.736831 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29f42f-6f37-4586-bd2d-722d74a05159 None admin] RBAC: Adding query filter params (name=network) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:50.737142 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29f42f-6f37-4586-bd2d-722d74a05159 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:50.738634 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29f42f-6f37-4586-bd2d-722d74a05159 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:50.748706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 372/770] 10.222.0.67 () {56 vars in 1112 bytes} [Tue May 31 14:25:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:50.750154 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14c427-f076-4b3b-b9cf-a7eb6da4d11c None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:50.908367 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:50.915340 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14c427-f076-4b3b-b9cf-a7eb6da4d11c None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:50.966344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0cf3c-6b2e-4b4e-9a82-648cc600c26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:50.966960 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0cf3c-6b2e-4b4e-9a82-648cc600c26f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:50.967249 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0cf3c-6b2e-4b4e-9a82-648cc600c26f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:50.967525 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0cf3c-6b2e-4b4e-9a82-648cc600c26f None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:50.968307 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd0cf3c-6b2e-4b4e-9a82-648cc600c26f None admin] RBAC: Adding query filter params (type=network) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:50.968620 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd0cf3c-6b2e-4b4e-9a82-648cc600c26f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:50.970062 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd0cf3c-6b2e-4b4e-9a82-648cc600c26f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:50.980011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 399/771] 10.222.0.67 () {56 vars in 1112 bytes} [Tue May 31 14:25:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:50.985495 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29f42f-6f37-4586-bd2d-722d74a05159 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:51.156376 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:51.160232 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29f42f-6f37-4586-bd2d-722d74a05159 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:51.215528 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a588c230-ad13-4423-af5c-d5640a78e9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:51.216212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588c230-ad13-4423-af5c-d5640a78e9f7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:51.216557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588c230-ad13-4423-af5c-d5640a78e9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:51.216781 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588c230-ad13-4423-af5c-d5640a78e9f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:51.217582 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a588c230-ad13-4423-af5c-d5640a78e9f7 None admin] RBAC: Adding query filter params (interface=public, service_id=2e066078442f47e6a51d9e1752f3c92b, region_id=RegionOne) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:51.217902 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a588c230-ad13-4423-af5c-d5640a78e9f7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:51.219293 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a588c230-ad13-4423-af5c-d5640a78e9f7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:51.232794 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 373/772] 10.222.0.67 () {56 vars in 1251 bytes} [Tue May 31 14:25:50 2022] GET /identity/v3/endpoints?service_id=2e066078442f47e6a51d9e1752f3c92b&interface=public®ion_id=RegionOne => generated 194 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:51.238587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0cf3c-6b2e-4b4e-9a82-648cc600c26f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:51.247113 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d32f8d-3763-43e7-b32d-8a08c01431b4 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:51.247404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d32f8d-3763-43e7-b32d-8a08c01431b4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:51.247684 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d32f8d-3763-43e7-b32d-8a08c01431b4 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:51.249314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 374/773] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 31 14:25:51.254296 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf40eb0-61ec-4cae-b140-80aabc525e38 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:51.254594 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf40eb0-61ec-4cae-b140-80aabc525e38 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:51.254912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf40eb0-61ec-4cae-b140-80aabc525e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:51.275725 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6bf40eb0-61ec-4cae-b140-80aabc525e38 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:51.276967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6bf40eb0-61ec-4cae-b140-80aabc525e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:51.400201 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 375/774] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 31 14:25:51.424682 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf40eb0-61ec-4cae-b140-80aabc525e38 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:51.437204 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0cf3c-6b2e-4b4e-9a82-648cc600c26f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:51.504972 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93d837-68ae-46d3-a4af-737a07e62b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:51.505546 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93d837-68ae-46d3-a4af-737a07e62b29 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:51.505829 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93d837-68ae-46d3-a4af-737a07e62b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:51.506108 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93d837-68ae-46d3-a4af-737a07e62b29 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:51.506833 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93d837-68ae-46d3-a4af-737a07e62b29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:51.508227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93d837-68ae-46d3-a4af-737a07e62b29 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:51.519726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 400/775] 10.222.0.67 () {56 vars in 1087 bytes} [Tue May 31 14:25:51 2022] GET /identity/v3/services => generated 772 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:51.558123 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:51.593143 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf40eb0-61ec-4cae-b140-80aabc525e38 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:51.684560 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbaf2a-beb2-483e-93ec-67b4c88a87c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:51.685409 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbaf2a-beb2-483e-93ec-67b4c88a87c2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:51.685693 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbaf2a-beb2-483e-93ec-67b4c88a87c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:51.685970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbaf2a-beb2-483e-93ec-67b4c88a87c2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:51.686629 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edbaf2a-beb2-483e-93ec-67b4c88a87c2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:51.688063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edbaf2a-beb2-483e-93ec-67b4c88a87c2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:51.706851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-1edbaf2a-beb2-483e-93ec-67b4c88a87c2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 31 14:25:51.706851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:51.706851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:25:51.706851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:25:51.706851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:25:51.706851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:25:51.706851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:25:51.706851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:25:51.706851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 31 14:25:51.706851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:51.706851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:25:51.706851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:51.706851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:51.706851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:51.706851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:51.706851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:51.707877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:51.707877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:51.707877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:51.707877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:51.707877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:51.707877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:51.707877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:51.707877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:25:51.707877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:25:51.707877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:25:51.707877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:25:51.707877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:51.707877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:51.707877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:51.707877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:51.707877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:51.708703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:51.708703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:51.708703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:51.708703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:51.708703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:51.708703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:51.708703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:51.708703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:51.708703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:51.708703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:51.708703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:51.708703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:25:51.708703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:25:51.708703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 31 14:25:51.708703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:51.719877 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93d837-68ae-46d3-a4af-737a07e62b29 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:51.722689 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 376/776] 10.222.0.67 () {56 vars in 1129 bytes} [Tue May 31 14:25:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:51.742552 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40027c25-bf5a-4dc3-9ace-dee28297180a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:51.742913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40027c25-bf5a-4dc3-9ace-dee28297180a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:51.743136 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40027c25-bf5a-4dc3-9ace-dee28297180a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:51.744687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 377/777] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 31 14:25:51.750665 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77056557-de15-41f1-ab37-a77b584f6574 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:51.751230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77056557-de15-41f1-ab37-a77b584f6574 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:51.751740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77056557-de15-41f1-ab37-a77b584f6574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:51.771136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:51.775447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-77056557-de15-41f1-ab37-a77b584f6574 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:51.776831 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-77056557-de15-41f1-ab37-a77b584f6574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:51.871867 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93d837-68ae-46d3-a4af-737a07e62b29 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:51.875727 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 378/778] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 31 14:25:51.899554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77056557-de15-41f1-ab37-a77b584f6574 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:51.913671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ebdc88-52c1-4e5c-a00c-a8b546dad86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:51.914637 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebdc88-52c1-4e5c-a00c-a8b546dad86b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:51.914950 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebdc88-52c1-4e5c-a00c-a8b546dad86b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:51.915233 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebdc88-52c1-4e5c-a00c-a8b546dad86b None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:51.916129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ebdc88-52c1-4e5c-a00c-a8b546dad86b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:51.916831 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ebdc88-52c1-4e5c-a00c-a8b546dad86b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:51.918510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ebdc88-52c1-4e5c-a00c-a8b546dad86b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:51.924741 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 401/779] 10.222.0.67 () {56 vars in 1124 bytes} [Tue May 31 14:25:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:51.930291 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ebdc88-52c1-4e5c-a00c-a8b546dad86b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:52.007333 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:52.021281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77056557-de15-41f1-ab37-a77b584f6574 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:52.040173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:52.057664 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ebdc88-52c1-4e5c-a00c-a8b546dad86b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:52.057891 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ecd866-2acf-424c-af5d-0cbb57fbd0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:52.058768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecd866-2acf-424c-af5d-0cbb57fbd0cf None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:52.059159 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecd866-2acf-424c-af5d-0cbb57fbd0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:52.059541 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecd866-2acf-424c-af5d-0cbb57fbd0cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:52.070335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ecd866-2acf-424c-af5d-0cbb57fbd0cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:52.082107 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-b7ecd866-2acf-424c-af5d-0cbb57fbd0cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:52.082107 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:52.082107 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:52.082107 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:52.082107 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:52.082107 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:52.082107 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:52.082107 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:52.082107 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:52.082107 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:52.082107 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:52.082107 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:52.082107 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:52.082107 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:52.082107 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:52.082107 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:52.083586 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:52.083586 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:52.083586 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:52.083586 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:52.083586 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:52.083586 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:52.083586 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:52.083586 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:52.083586 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:52.083586 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:52.083586 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:52.083586 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:52.083586 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:52.083586 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:52.083586 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:52.083586 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:52.084580 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:52.084580 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:52.084580 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:52.084580 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:52.084580 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:52.084580 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:52.084580 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:52.084580 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:52.084580 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:52.084580 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:52.084580 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:52.084580 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:52.085657 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 379/780] 10.222.0.67 () {56 vars in 1102 bytes} [Tue May 31 14:25:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:52.092096 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3688107-de66-4626-a052-a06f8a427763 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:52.092473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3688107-de66-4626-a052-a06f8a427763 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:52.092850 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3688107-de66-4626-a052-a06f8a427763 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:52.098623 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 380/781] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:52.098623 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3688107-de66-4626-a052-a06f8a427763 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:52.109584 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711fd5d-7a46-43dd-997c-3a320e9ebb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:52.110671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711fd5d-7a46-43dd-997c-3a320e9ebb8f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:52.111144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711fd5d-7a46-43dd-997c-3a320e9ebb8f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:52.111637 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711fd5d-7a46-43dd-997c-3a320e9ebb8f None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:52.112557 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a711fd5d-7a46-43dd-997c-3a320e9ebb8f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:52.113168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a711fd5d-7a46-43dd-997c-3a320e9ebb8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:52.114939 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a711fd5d-7a46-43dd-997c-3a320e9ebb8f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:52.120843 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 402/782] 10.222.0.67 () {56 vars in 1124 bytes} [Tue May 31 14:25:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:52.122324 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e94ef-1a64-4cc6-a461-9e0ac1ef1dfd None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:52.122856 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e94ef-1a64-4cc6-a461-9e0ac1ef1dfd None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:52.123397 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e94ef-1a64-4cc6-a461-9e0ac1ef1dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:52.142829 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-ca9e94ef-1a64-4cc6-a461-9e0ac1ef1dfd None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:52.144493 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ca9e94ef-1a64-4cc6-a461-9e0ac1ef1dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:52.208658 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3688107-de66-4626-a052-a06f8a427763 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:52.224895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 403/783] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:25:52.226215 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e94ef-1a64-4cc6-a461-9e0ac1ef1dfd None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:52.249769 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bb050-8894-4072-b69b-211a69a1b509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:52.249769 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bb050-8894-4072-b69b-211a69a1b509 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:52.249769 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bb050-8894-4072-b69b-211a69a1b509 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:52.249769 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bb050-8894-4072-b69b-211a69a1b509 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:52.250180 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bb050-8894-4072-b69b-211a69a1b509 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:52.250180 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bb050-8894-4072-b69b-211a69a1b509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:52.250180 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bb050-8894-4072-b69b-211a69a1b509 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:52.257879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 381/784] 10.222.0.67 () {56 vars in 1103 bytes} [Tue May 31 14:25:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:25:52.264527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bb050-8894-4072-b69b-211a69a1b509 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:52.278205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:52.337068 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:52.364557 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e94ef-1a64-4cc6-a461-9e0ac1ef1dfd None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:52.374688 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bb050-8894-4072-b69b-211a69a1b509 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:52.413489 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18b14b-c2dd-454e-b43e-cb43fa0af029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:52.414841 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18b14b-c2dd-454e-b43e-cb43fa0af029 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:52.415462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18b14b-c2dd-454e-b43e-cb43fa0af029 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:52.415958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18b14b-c2dd-454e-b43e-cb43fa0af029 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:52.416169 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77e32f-b58f-488d-936a-fcb09dad4cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:52.417010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77e32f-b58f-488d-936a-fcb09dad4cdf None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:52.417297 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77e32f-b58f-488d-936a-fcb09dad4cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:52.417577 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77e32f-b58f-488d-936a-fcb09dad4cdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:52.417864 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18b14b-c2dd-454e-b43e-cb43fa0af029 None admin] RBAC: Adding query filter params (interface=public, service_id=628759c1632340f6a176fa1da5069289, region_id=RegionOne) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:52.418379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18b14b-c2dd-454e-b43e-cb43fa0af029 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:52.420194 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18b14b-c2dd-454e-b43e-cb43fa0af029 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:52.428770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b77e32f-b58f-488d-936a-fcb09dad4cdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:52.428917 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 404/785] 10.222.0.67 () {56 vars in 1251 bytes} [Tue May 31 14:25:52 2022] GET /identity/v3/endpoints?service_id=628759c1632340f6a176fa1da5069289&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:52.430015 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18b14b-c2dd-454e-b43e-cb43fa0af029 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:52.430276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b77e32f-b58f-488d-936a-fcb09dad4cdf None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:52.449893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-4b77e32f-b58f-488d-936a-fcb09dad4cdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:52.449893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:52.449893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:52.449893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:52.449893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:52.449893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:52.449893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:52.449893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:52.449893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:52.449893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:52.449893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:52.449893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:52.449893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:52.449893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:52.449893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:52.451212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:52.451212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:52.451212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:52.451212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:52.451212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:52.451212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:52.451212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:52.451212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:52.451212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:52.451212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:52.451212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:52.451212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:52.451212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:52.451212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:52.451212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:52.451212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:52.451212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:52.452161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:52.452161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:52.452161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:52.452161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:52.456259 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 382/786] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:25:52.456988 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77e32f-b58f-488d-936a-fcb09dad4cdf None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:52.525122 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:52.548652 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18b14b-c2dd-454e-b43e-cb43fa0af029 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:52.569221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77e32f-b58f-488d-936a-fcb09dad4cdf None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:52.588514 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940ab97-290e-408b-b2e4-b72c6a584aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:52.589673 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940ab97-290e-408b-b2e4-b72c6a584aae None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:52.590192 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940ab97-290e-408b-b2e4-b72c6a584aae None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:52.591238 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940ab97-290e-408b-b2e4-b72c6a584aae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:52.605578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2940ab97-290e-408b-b2e4-b72c6a584aae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:52.605744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0107c-35c4-4ed9-b15d-8c069a5debb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:52.606388 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0107c-35c4-4ed9-b15d-8c069a5debb7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:52.606699 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0107c-35c4-4ed9-b15d-8c069a5debb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:52.606982 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0107c-35c4-4ed9-b15d-8c069a5debb7 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:52.607672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0107c-35c4-4ed9-b15d-8c069a5debb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:52.609051 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0107c-35c4-4ed9-b15d-8c069a5debb7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:52.614942 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 383/787] 10.222.0.67 () {56 vars in 1087 bytes} [Tue May 31 14:25:52 2022] GET /identity/v3/services => generated 772 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:52.615822 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0107c-35c4-4ed9-b15d-8c069a5debb7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:52.617085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-2940ab97-290e-408b-b2e4-b72c6a584aae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:25:52.617085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:52.617085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:52.617085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:52.617085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:52.617085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:52.617085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:52.617085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:52.617085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:52.617085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:52.617085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:52.617085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:52.617085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:52.617085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:52.617085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:52.617085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:52.618084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:52.618084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:52.618084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:52.618084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:52.618084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:52.618084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:52.618084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:52.618084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:52.618084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:52.618084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:52.618084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:52.618084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:52.618084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:52.618084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:52.618084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:52.618084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:52.619023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:52.619023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:52.619023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:52.619023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:52.619023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:52.619023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:52.619023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:52.619023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:52.619023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:52.619023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:52.619023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:25:52.619023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:52.626975 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 405/788] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:25:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:52.631845 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940ab97-290e-408b-b2e4-b72c6a584aae None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:52.667015 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:52.800903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940ab97-290e-408b-b2e4-b72c6a584aae None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:52.840113 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d047b7-fbcf-4e58-a460-ef95466eb1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:52.840830 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d047b7-fbcf-4e58-a460-ef95466eb1f5 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:52.841541 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d047b7-fbcf-4e58-a460-ef95466eb1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:52.841825 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d047b7-fbcf-4e58-a460-ef95466eb1f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:52.842595 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d047b7-fbcf-4e58-a460-ef95466eb1f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:52.842937 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d047b7-fbcf-4e58-a460-ef95466eb1f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:52.844637 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d047b7-fbcf-4e58-a460-ef95466eb1f5 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:52.856625 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 406/789] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:25:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:52.862165 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d047b7-fbcf-4e58-a460-ef95466eb1f5 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:52.891259 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0107c-35c4-4ed9-b15d-8c069a5debb7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:52.925697 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:52.942357 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744dfe39-16fa-44f2-8a22-e8671964dece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:52.943095 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744dfe39-16fa-44f2-8a22-e8671964dece None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:52.943386 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744dfe39-16fa-44f2-8a22-e8671964dece None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:52.943668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744dfe39-16fa-44f2-8a22-e8671964dece None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:52.944698 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744dfe39-16fa-44f2-8a22-e8671964dece None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:52.945412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744dfe39-16fa-44f2-8a22-e8671964dece None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:52.947102 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744dfe39-16fa-44f2-8a22-e8671964dece None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:52.950563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:52.965610 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 384/790] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:52.967074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7d3f5-a50e-4ac8-ac5c-bac6ce37a0f1 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:52.967368 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7d3f5-a50e-4ac8-ac5c-bac6ce37a0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:52.967913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7d3f5-a50e-4ac8-ac5c-bac6ce37a0f1 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:52.969958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 385/791] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:52.971079 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7d3f5-a50e-4ac8-ac5c-bac6ce37a0f1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:52.992493 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d047b7-fbcf-4e58-a460-ef95466eb1f5 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:53.032087 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7653912-5cbe-449b-8665-ac022980a305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:53.032901 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7653912-5cbe-449b-8665-ac022980a305 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:53.033196 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7653912-5cbe-449b-8665-ac022980a305 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:53.033478 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7653912-5cbe-449b-8665-ac022980a305 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:53.053178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7653912-5cbe-449b-8665-ac022980a305 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:53.056177 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7653912-5cbe-449b-8665-ac022980a305 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:53.076755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-a7653912-5cbe-449b-8665-ac022980a305 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:53.076755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:53.076755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:53.076755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:53.076755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:53.076755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:53.076755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:53.076755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:53.076755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:53.076755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:53.076755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:53.076755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:53.076755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:53.076755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:53.076755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:53.077739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:53.077739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:53.077739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:53.077739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:53.077739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:53.077739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:53.077739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:53.077739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:53.077739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:53.077739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:53.077739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:53.077739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:53.077739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:53.077739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:53.077739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:53.077739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:53.077739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:53.078631 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:53.078631 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:53.078631 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:53.078631 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:53.086268 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 407/792] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:25:53.088002 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6564de-9b90-4fca-9539-0a21ad29cb66 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:53.089063 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6564de-9b90-4fca-9539-0a21ad29cb66 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:53.089541 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6564de-9b90-4fca-9539-0a21ad29cb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:53.110214 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7d3f5-a50e-4ac8-ac5c-bac6ce37a0f1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:53.112006 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6e6564de-9b90-4fca-9539-0a21ad29cb66 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:53.113462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6e6564de-9b90-4fca-9539-0a21ad29cb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:53.152730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b6528-b613-40e6-939a-87a998231bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:53.153863 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b6528-b613-40e6-939a-87a998231bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:53.154344 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b6528-b613-40e6-939a-87a998231bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:53.154868 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b6528-b613-40e6-939a-87a998231bc3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:53.164768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b6528-b613-40e6-939a-87a998231bc3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:53.166432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b6528-b613-40e6-939a-87a998231bc3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:53.186400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-4a8b6528-b613-40e6-939a-87a998231bc3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 31 14:25:53.186400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:53.186400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:53.186400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:53.186400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:53.186400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:53.186400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:53.186400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:53.186400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:53.186400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:53.186400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:53.186400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:53.186400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:25:53.186400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:25:53.186400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:25:53.186400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:25:53.187557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:53.187557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:53.187557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:25:53.187557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:53.187557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:25:53.187557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:53.187557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:53.187557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:53.187557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:53.187557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:53.187557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:53.187557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:53.187557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:53.187557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:53.187557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:53.187557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:53.187557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:53.187557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:53.188706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:53.188706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:53.188706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:25:53.188706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:25:53.188706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:25:53.188706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:25:53.188706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:25:53.188706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:25:53.188706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 31 14:25:53.188706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:53.191415 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 386/793] 10.222.0.67 () {56 vars in 1117 bytes} [Tue May 31 14:25:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:53.192703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b6528-b613-40e6-939a-87a998231bc3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:53.205870 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 408/794] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:25:53.207324 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6564de-9b90-4fca-9539-0a21ad29cb66 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:53.252429 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:53.274385 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:53.321388 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b6528-b613-40e6-939a-87a998231bc3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:53.342093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6564de-9b90-4fca-9539-0a21ad29cb66 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:53.361120 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a11ab-7e1b-4dd4-a3f8-70db4ce5d44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:53.362277 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a11ab-7e1b-4dd4-a3f8-70db4ce5d44c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:53.362799 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a11ab-7e1b-4dd4-a3f8-70db4ce5d44c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:53.363093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a11ab-7e1b-4dd4-a3f8-70db4ce5d44c None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:53.363846 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a11ab-7e1b-4dd4-a3f8-70db4ce5d44c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:53.364137 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a11ab-7e1b-4dd4-a3f8-70db4ce5d44c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:53.365514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a11ab-7e1b-4dd4-a3f8-70db4ce5d44c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:53.384215 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 387/795] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:53.384215 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a11ab-7e1b-4dd4-a3f8-70db4ce5d44c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:53.392684 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7676a-4a0d-4374-ba7e-e638ba6f94ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:53.394187 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7676a-4a0d-4374-ba7e-e638ba6f94ff None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:53.394701 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7676a-4a0d-4374-ba7e-e638ba6f94ff None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:53.395140 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7676a-4a0d-4374-ba7e-e638ba6f94ff None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:53.396601 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7676a-4a0d-4374-ba7e-e638ba6f94ff None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:53.397060 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7676a-4a0d-4374-ba7e-e638ba6f94ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:53.399375 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7676a-4a0d-4374-ba7e-e638ba6f94ff None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:53.420639 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 409/796] 10.222.0.67 () {56 vars in 1149 bytes} [Tue May 31 14:25:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:53.421610 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7676a-4a0d-4374-ba7e-e638ba6f94ff None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:53.503516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a11ab-7e1b-4dd4-a3f8-70db4ce5d44c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:53.511447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:53.534263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7676a-4a0d-4374-ba7e-e638ba6f94ff None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:53.541482 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af53b1f-c65a-4d8e-a65d-3f99a6fd98da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:53.542513 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af53b1f-c65a-4d8e-a65d-3f99a6fd98da None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:53.542826 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af53b1f-c65a-4d8e-a65d-3f99a6fd98da None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:53.543101 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af53b1f-c65a-4d8e-a65d-3f99a6fd98da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:53.546506 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:53.558066 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af53b1f-c65a-4d8e-a65d-3f99a6fd98da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:53.569447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-7af53b1f-c65a-4d8e-a65d-3f99a6fd98da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:25:53.569447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:53.569447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:53.569447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:53.569447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:53.569447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:53.569447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:53.569447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:53.569447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:53.569447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:53.569447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:53.569447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:53.569447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:53.569447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:53.569447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:53.569447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:53.570394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:53.570394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:53.570394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:53.570394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:53.570394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:53.570394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:53.570394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:53.570394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:53.570394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:53.570394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:53.570394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:53.570394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:53.570394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:53.570394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:53.570394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:53.570394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:53.571259 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:53.571259 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:53.571259 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:53.571259 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:53.571259 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:53.571259 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:53.571259 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:53.571259 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:53.571259 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:53.571259 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:53.571259 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:25:53.571259 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:53.572674 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 388/797] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:25:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:53.573807 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af53b1f-c65a-4d8e-a65d-3f99a6fd98da None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:53.578902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e598238-ac5c-407f-8c0d-b8faaf0f178d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:53.579770 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e598238-ac5c-407f-8c0d-b8faaf0f178d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:53.580093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e598238-ac5c-407f-8c0d-b8faaf0f178d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:53.580386 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e598238-ac5c-407f-8c0d-b8faaf0f178d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:53.590192 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e598238-ac5c-407f-8c0d-b8faaf0f178d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:53.591904 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e598238-ac5c-407f-8c0d-b8faaf0f178d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:53.607482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-3e598238-ac5c-407f-8c0d-b8faaf0f178d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 31 14:25:53.607482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:53.607482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:53.607482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:53.607482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:53.607482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:53.607482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:53.607482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:53.607482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:53.607482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:53.607482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:53.607482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:53.607482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:25:53.607482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:25:53.607482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:25:53.607482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:25:53.608464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:53.608464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:53.608464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:25:53.608464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:53.608464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:25:53.608464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:53.608464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:53.608464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:53.608464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:53.608464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:53.608464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:53.608464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:53.608464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:53.608464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:53.608464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:53.608464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:53.608464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:53.608464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:53.609417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:53.609417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:53.609417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:25:53.609417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:25:53.609417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:25:53.609417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:25:53.609417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:25:53.609417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:25:53.609417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 31 14:25:53.609417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:53.613582 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 410/798] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:25:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:53.614482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e598238-ac5c-407f-8c0d-b8faaf0f178d None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:53.711078 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af53b1f-c65a-4d8e-a65d-3f99a6fd98da None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:53.735602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e598238-ac5c-407f-8c0d-b8faaf0f178d None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:53.756958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20978f3-bc63-4682-a064-9a5d47166c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:53.758537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20978f3-bc63-4682-a064-9a5d47166c7e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:53.759078 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20978f3-bc63-4682-a064-9a5d47166c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:53.759556 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20978f3-bc63-4682-a064-9a5d47166c7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:53.776832 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20978f3-bc63-4682-a064-9a5d47166c7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:53.779351 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20978f3-bc63-4682-a064-9a5d47166c7e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:53.785221 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb2e0e-131c-4388-96b8-092efc973648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:53.785898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb2e0e-131c-4388-96b8-092efc973648 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:53.786186 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb2e0e-131c-4388-96b8-092efc973648 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:53.786468 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb2e0e-131c-4388-96b8-092efc973648 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:53.787280 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb2e0e-131c-4388-96b8-092efc973648 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:53.787598 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb2e0e-131c-4388-96b8-092efc973648 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:53.788962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb2e0e-131c-4388-96b8-092efc973648 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:53.800638 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 411/799] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:25:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:53.801801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-b20978f3-bc63-4682-a064-9a5d47166c7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:53.801801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:53.801801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:53.801801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:53.801801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:53.801801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:53.801801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:53.801801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:53.801801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:53.801801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:53.801801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:53.801801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:53.801801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:53.801801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:53.801801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:53.802692 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:53.802692 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:53.802692 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:53.802692 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:53.802692 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:53.802692 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:53.802692 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:53.802692 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:53.802692 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:53.802692 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:53.802692 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:53.802692 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:53.802692 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:53.802692 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:53.802692 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:53.802692 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:53.802692 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:53.804061 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:53.804061 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:53.804061 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:53.804061 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:53.804441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb2e0e-131c-4388-96b8-092efc973648 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:53.804528 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 389/800] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:53.806689 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20978f3-bc63-4682-a064-9a5d47166c7e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:53.831095 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:53.840993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:53.914406 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb2e0e-131c-4388-96b8-092efc973648 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:53.926344 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20978f3-bc63-4682-a064-9a5d47166c7e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:53.951853 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d82bd-7f4e-43b8-8924-e9418fe37df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:53.952564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d82bd-7f4e-43b8-8924-e9418fe37df7 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:53.952851 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d82bd-7f4e-43b8-8924-e9418fe37df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:53.953133 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d82bd-7f4e-43b8-8924-e9418fe37df7 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:53.953988 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d82bd-7f4e-43b8-8924-e9418fe37df7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:53.954293 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d82bd-7f4e-43b8-8924-e9418fe37df7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:53.955951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d82bd-7f4e-43b8-8924-e9418fe37df7 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:53.967064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb1a06-5a06-43c1-bf9a-7f7e42cbed20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:53.968185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb1a06-5a06-43c1-bf9a-7f7e42cbed20 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:53.968672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb1a06-5a06-43c1-bf9a-7f7e42cbed20 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:53.969151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb1a06-5a06-43c1-bf9a-7f7e42cbed20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:53.979025 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 412/801] 10.222.0.67 () {56 vars in 1135 bytes} [Tue May 31 14:25:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:53.979870 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d82bd-7f4e-43b8-8924-e9418fe37df7 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:53.981634 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb1a06-5a06-43c1-bf9a-7f7e42cbed20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:53.983488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb1a06-5a06-43c1-bf9a-7f7e42cbed20 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:54.003166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-4bdb1a06-5a06-43c1-bf9a-7f7e42cbed20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:54.003166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:54.003166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:54.003166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:54.003166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:54.003166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:54.003166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:54.003166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:54.003166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:54.003166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:54.003166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:54.003166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:54.003166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:54.003166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:54.003166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:54.004106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:54.004106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:54.004106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:54.004106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:54.004106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:54.004106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:54.004106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:54.004106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:54.004106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:54.004106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:54.004106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:54.004106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:54.004106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:54.004106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:54.004106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:54.004106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:54.004106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:54.005020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:54.005020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:54.005020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:54.005020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:54.012132 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 390/802] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:54.013336 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb1a06-5a06-43c1-bf9a-7f7e42cbed20 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:54.085626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:54.098124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:54.099786 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d82bd-7f4e-43b8-8924-e9418fe37df7 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:54.143174 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5484d6-ef24-44bd-8fe5-b98979edea12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:54.143857 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5484d6-ef24-44bd-8fe5-b98979edea12 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:54.144132 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5484d6-ef24-44bd-8fe5-b98979edea12 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:54.144390 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5484d6-ef24-44bd-8fe5-b98979edea12 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:54.145120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5484d6-ef24-44bd-8fe5-b98979edea12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:54.145427 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5484d6-ef24-44bd-8fe5-b98979edea12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:54.146091 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb1a06-5a06-43c1-bf9a-7f7e42cbed20 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:54.146818 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5484d6-ef24-44bd-8fe5-b98979edea12 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:54.159075 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 413/803] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:54.160137 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5484d6-ef24-44bd-8fe5-b98979edea12 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:54.186971 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186cdd8-336a-45e1-ad69-0eba5cc81f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:54.187784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186cdd8-336a-45e1-ad69-0eba5cc81f98 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:54.188055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186cdd8-336a-45e1-ad69-0eba5cc81f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:54.188519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186cdd8-336a-45e1-ad69-0eba5cc81f98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:54.200280 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186cdd8-336a-45e1-ad69-0eba5cc81f98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:54.201864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186cdd8-336a-45e1-ad69-0eba5cc81f98 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:54.219710 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-c186cdd8-336a-45e1-ad69-0eba5cc81f98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:54.219710 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:54.219710 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:54.219710 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:54.219710 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:54.219710 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:54.219710 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:54.219710 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:54.219710 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:54.219710 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:54.219710 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:54.219710 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:54.219710 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:54.219710 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:54.219710 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:54.220591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:54.220591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:54.220591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:54.220591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:54.220591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:54.220591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:54.220591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:54.220591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:54.220591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:54.220591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:54.220591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:54.220591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:54.220591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:54.220591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:54.220591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:54.220591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:54.220591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:54.221493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:54.221493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:54.221493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:54.221493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:54.221493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 391/804] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:54.222199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb18de-eee3-4e3b-87c3-8f9dc9faa7be None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:54.222199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb18de-eee3-4e3b-87c3-8f9dc9faa7be None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:54.222457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb18de-eee3-4e3b-87c3-8f9dc9faa7be None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:54.223921 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 392/805] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:54.227918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb18de-eee3-4e3b-87c3-8f9dc9faa7be None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:54.268789 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5484d6-ef24-44bd-8fe5-b98979edea12 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:54.304871 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362465c-e3ed-4391-bc37-e94066f89106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:54.305535 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362465c-e3ed-4391-bc37-e94066f89106 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:54.305817 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362465c-e3ed-4391-bc37-e94066f89106 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:54.306104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362465c-e3ed-4391-bc37-e94066f89106 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:54.306844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362465c-e3ed-4391-bc37-e94066f89106 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:54.307187 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362465c-e3ed-4391-bc37-e94066f89106 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:54.309382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362465c-e3ed-4391-bc37-e94066f89106 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:54.326917 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 414/806] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:54.326917 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362465c-e3ed-4391-bc37-e94066f89106 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:54.338222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:54.346635 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb18de-eee3-4e3b-87c3-8f9dc9faa7be None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:54.372320 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:54.385867 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38882270-1b8c-43e2-8ddd-034929c18188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:54.386980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38882270-1b8c-43e2-8ddd-034929c18188 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:54.387272 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38882270-1b8c-43e2-8ddd-034929c18188 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:54.387554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38882270-1b8c-43e2-8ddd-034929c18188 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:54.394595 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38882270-1b8c-43e2-8ddd-034929c18188 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:54.396060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38882270-1b8c-43e2-8ddd-034929c18188 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:54.411466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-38882270-1b8c-43e2-8ddd-034929c18188 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:54.411466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:54.411466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:54.411466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:54.411466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:54.411466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:54.411466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:54.411466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:54.411466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:54.411466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:54.411466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:54.411466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:54.411466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:25:54.411466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:25:54.411466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:25:54.413261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:25:54.413261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:54.413261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:54.413261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:54.413261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:54.413261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:54.413261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:54.413261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:54.413261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:54.413261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:54.413261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:54.413261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:54.413261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:54.413261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:54.413261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:54.413261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:54.413261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:54.415050 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:25:54.415050 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:25:54.415050 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:25:54.415050 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:25:54.415050 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:25:54.415050 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:25:54.415050 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:54.415050 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:54.418349 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 393/807] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:54.420082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d555b2d-752c-4204-a2f6-b457cfaf1b24 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:54.420380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d555b2d-752c-4204-a2f6-b457cfaf1b24 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:54.420667 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d555b2d-752c-4204-a2f6-b457cfaf1b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:54.436894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362465c-e3ed-4391-bc37-e94066f89106 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:54.438838 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2d555b2d-752c-4204-a2f6-b457cfaf1b24 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:54.440148 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2d555b2d-752c-4204-a2f6-b457cfaf1b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:54.470603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eec2de-e63a-405c-806c-4e343a815ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:54.471296 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eec2de-e63a-405c-806c-4e343a815ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:54.471579 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eec2de-e63a-405c-806c-4e343a815ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:54.471860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eec2de-e63a-405c-806c-4e343a815ef4 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:54.472581 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eec2de-e63a-405c-806c-4e343a815ef4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:54.472889 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eec2de-e63a-405c-806c-4e343a815ef4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:54.474272 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eec2de-e63a-405c-806c-4e343a815ef4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:54.486147 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 415/808] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:54.486832 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eec2de-e63a-405c-806c-4e343a815ef4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:54.517581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 394/809] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:25:54.518782 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d555b2d-752c-4204-a2f6-b457cfaf1b24 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:54.562913 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:54.600237 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eec2de-e63a-405c-806c-4e343a815ef4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:54.637868 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d2581-e366-4170-9d8d-0a5ec14162e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:54.638681 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d2581-e366-4170-9d8d-0a5ec14162e4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:54.638973 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d2581-e366-4170-9d8d-0a5ec14162e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:54.639257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d2581-e366-4170-9d8d-0a5ec14162e4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:54.642052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d555b2d-752c-4204-a2f6-b457cfaf1b24 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:54.647009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d2581-e366-4170-9d8d-0a5ec14162e4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:54.648462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d2581-e366-4170-9d8d-0a5ec14162e4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:54.657147 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:54.663715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-1a1d2581-e366-4170-9d8d-0a5ec14162e4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 31 14:25:54.663715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:54.663715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:54.663715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:54.663715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:54.663715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:54.663715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:54.663715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:54.663715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:54.663715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:54.663715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:54.663715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:54.663715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:25:54.663715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:25:54.663715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:25:54.663715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:25:54.664671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:54.664671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:54.664671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:25:54.664671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:54.664671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:25:54.664671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:54.664671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:54.664671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:54.664671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:54.664671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:54.664671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:54.664671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:54.664671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:54.664671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:54.664671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:54.664671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:54.664671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:54.664671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:54.665652 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:54.665652 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:54.665652 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:25:54.665652 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:25:54.665652 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:25:54.665652 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:25:54.665652 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:25:54.665652 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:25:54.665652 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 31 14:25:54.665652 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:54.670160 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 416/810] 10.222.0.67 () {56 vars in 1093 bytes} [Tue May 31 14:25:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:54.671086 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d2581-e366-4170-9d8d-0a5ec14162e4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:54.681351 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f131af-2f08-4214-9f33-34b7fd090d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:54.682024 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f131af-2f08-4214-9f33-34b7fd090d77 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:54.682350 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f131af-2f08-4214-9f33-34b7fd090d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:54.682704 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f131af-2f08-4214-9f33-34b7fd090d77 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:54.683522 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f131af-2f08-4214-9f33-34b7fd090d77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:54.683837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f131af-2f08-4214-9f33-34b7fd090d77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:54.685268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f131af-2f08-4214-9f33-34b7fd090d77 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:54.702809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 395/811] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:25:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:54.702809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f131af-2f08-4214-9f33-34b7fd090d77 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:54.789149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d2581-e366-4170-9d8d-0a5ec14162e4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:54.810709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f131af-2f08-4214-9f33-34b7fd090d77 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:54.819688 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:54.829315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace79511-289b-42f1-a4d7-f95d7d0400a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:54.830112 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace79511-289b-42f1-a4d7-f95d7d0400a4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:54.830396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace79511-289b-42f1-a4d7-f95d7d0400a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:54.830691 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace79511-289b-42f1-a4d7-f95d7d0400a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:54.837417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace79511-289b-42f1-a4d7-f95d7d0400a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:54.840028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace79511-289b-42f1-a4d7-f95d7d0400a4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:54.850710 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc367325-5302-43a7-801e-72a15deed405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:54.851546 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc367325-5302-43a7-801e-72a15deed405 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:54.851838 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc367325-5302-43a7-801e-72a15deed405 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:54.852116 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc367325-5302-43a7-801e-72a15deed405 None admin] PATH_INFO: `/v3/services/network` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:54.852777 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc367325-5302-43a7-801e-72a15deed405 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:54.854763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc367325-5302-43a7-801e-72a15deed405 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:54.854863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-ace79511-289b-42f1-a4d7-f95d7d0400a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:54.854863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:54.854863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:54.854863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:54.854863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:54.854863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:54.854863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:54.854863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:54.854863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:54.854863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:54.854863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:54.854863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:54.854863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:25:54.854863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:25:54.854863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:25:54.855719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:25:54.855719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:54.855719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:54.855719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:54.855719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:54.855719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:54.855719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:54.855719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:54.855719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:54.855719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:54.855719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:54.855719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:54.855719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:54.855719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:54.855719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:54.855719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:54.855719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:54.856607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:25:54.856607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:25:54.856607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:25:54.856607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:25:54.856607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:25:54.856607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:25:54.856607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:54.856607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:54.856607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 417/812] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:54.857089 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace79511-289b-42f1-a4d7-f95d7d0400a4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:54.871112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-cc367325-5302-43a7-801e-72a15deed405 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 31 14:25:54.871112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:54.871112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:25:54.871112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:25:54.871112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:25:54.871112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:25:54.871112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:25:54.871112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:25:54.871112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 31 14:25:54.871112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:54.871112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:25:54.871112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:54.871112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:54.871112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:54.871112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:54.871112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:54.871112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:54.872124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:54.872124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:54.872124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:54.872124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:54.872124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:54.872124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:54.872124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:25:54.872124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:25:54.872124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:25:54.872124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:25:54.872124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:54.872124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:54.872124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:54.872124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:54.872124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:54.872124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:54.872997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:54.872997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:54.872997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:54.872997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:54.872997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:54.872997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:54.872997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:54.872997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:54.872997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:54.872997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:54.872997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:25:54.872997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:25:54.872997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 31 14:25:54.872997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:54.872997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 396/813] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:54 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:54.873782 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc367325-5302-43a7-801e-72a15deed405 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:54.915146 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:54.980569 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace79511-289b-42f1-a4d7-f95d7d0400a4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:54.985483 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc367325-5302-43a7-801e-72a15deed405 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:55.019317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b195e-a9c5-465e-901e-9759e858856b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:55.019998 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b195e-a9c5-465e-901e-9759e858856b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:55.020289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b195e-a9c5-465e-901e-9759e858856b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:55.020568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b195e-a9c5-465e-901e-9759e858856b None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:55.021407 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b195e-a9c5-465e-901e-9759e858856b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:55.021724 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b195e-a9c5-465e-901e-9759e858856b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:55.022949 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c990cf-5903-41a5-8267-721c4321042f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:55.023192 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b195e-a9c5-465e-901e-9759e858856b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:55.024213 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c990cf-5903-41a5-8267-721c4321042f None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:55.024677 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c990cf-5903-41a5-8267-721c4321042f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:55.025136 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c990cf-5903-41a5-8267-721c4321042f None admin] PATH_INFO: `/v3/projects/f965a93a2f674598890bd8cafededb2c/users/0b7d100f2407432381d119c25814fff3/roles/5956aa478bfc4d03919395e637e61c83` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:55.040799 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c990cf-5903-41a5-8267-721c4321042f None admin] RBAC: Authorizing `identity:create_grant(project_id=f965a93a2f674598890bd8cafededb2c, user_id=0b7d100f2407432381d119c25814fff3, role_id=5956aa478bfc4d03919395e637e61c83)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:55.042488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c990cf-5903-41a5-8267-721c4321042f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:55.042597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 418/814] 10.222.0.67 () {56 vars in 1133 bytes} [Tue May 31 14:25:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:55.043380 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b195e-a9c5-465e-901e-9759e858856b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:55.057179 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 397/815] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:25:54 2022] PUT /identity/v3/projects/f965a93a2f674598890bd8cafededb2c/users/0b7d100f2407432381d119c25814fff3/roles/5956aa478bfc4d03919395e637e61c83 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:25:55.058043 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c990cf-5903-41a5-8267-721c4321042f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:55.102357 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:55.167799 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:55.173113 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b195e-a9c5-465e-901e-9759e858856b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:55.186274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c990cf-5903-41a5-8267-721c4321042f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:55.211547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716eb726-eab0-4500-8a24-3dae6c4eeb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:55.212241 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716eb726-eab0-4500-8a24-3dae6c4eeb8c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:55.212527 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716eb726-eab0-4500-8a24-3dae6c4eeb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:55.212825 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716eb726-eab0-4500-8a24-3dae6c4eeb8c None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:55.214181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716eb726-eab0-4500-8a24-3dae6c4eeb8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:55.214497 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716eb726-eab0-4500-8a24-3dae6c4eeb8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:55.215990 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716eb726-eab0-4500-8a24-3dae6c4eeb8c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:55.225895 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298474a8-0e47-4e88-b1b2-54ec4fc63099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:55.226826 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298474a8-0e47-4e88-b1b2-54ec4fc63099 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:55.227117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298474a8-0e47-4e88-b1b2-54ec4fc63099 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:55.227397 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298474a8-0e47-4e88-b1b2-54ec4fc63099 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:55.228123 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298474a8-0e47-4e88-b1b2-54ec4fc63099 None admin] RBAC: Adding query filter params (name=network) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:55.228435 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298474a8-0e47-4e88-b1b2-54ec4fc63099 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:55.229883 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298474a8-0e47-4e88-b1b2-54ec4fc63099 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:55.235531 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 398/816] 10.222.0.67 () {56 vars in 1112 bytes} [Tue May 31 14:25:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:55.236065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 419/817] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:25:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:55.236358 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298474a8-0e47-4e88-b1b2-54ec4fc63099 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:55.238042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c87603-d5f7-480a-9aac-23a96782fb4b None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:55.238512 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c87603-d5f7-480a-9aac-23a96782fb4b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:55.239015 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c87603-d5f7-480a-9aac-23a96782fb4b None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:55.241093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 420/818] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:55.241853 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c87603-d5f7-480a-9aac-23a96782fb4b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:55.350020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298474a8-0e47-4e88-b1b2-54ec4fc63099 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:55.353342 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c87603-d5f7-480a-9aac-23a96782fb4b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:55.360284 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:55.386887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b1632-557a-4f01-b94e-c8d18092db21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:55.387543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b1632-557a-4f01-b94e-c8d18092db21 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:55.387825 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b1632-557a-4f01-b94e-c8d18092db21 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:55.388099 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b1632-557a-4f01-b94e-c8d18092db21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:55.390304 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f8032-bcaa-42df-9b1a-7903a6881add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:55.391525 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f8032-bcaa-42df-9b1a-7903a6881add None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:55.391810 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f8032-bcaa-42df-9b1a-7903a6881add None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:55.392085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f8032-bcaa-42df-9b1a-7903a6881add None admin] PATH_INFO: `/v3/projects/f965a93a2f674598890bd8cafededb2c/users/dfa2282f957941b98db1d7497da3351b/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:55.394052 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f8032-bcaa-42df-9b1a-7903a6881add None admin] RBAC: Authorizing `identity:create_grant(project_id=f965a93a2f674598890bd8cafededb2c, user_id=dfa2282f957941b98db1d7497da3351b, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:55.395531 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f8032-bcaa-42df-9b1a-7903a6881add None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:55.407237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:55.418673 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 421/819] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:25:55 2022] PUT /identity/v3/projects/f965a93a2f674598890bd8cafededb2c/users/dfa2282f957941b98db1d7497da3351b/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:25:55.418673 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f8032-bcaa-42df-9b1a-7903a6881add None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:55.469626 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b1632-557a-4f01-b94e-c8d18092db21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:55.472224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b1632-557a-4f01-b94e-c8d18092db21 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:55.565663 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f8032-bcaa-42df-9b1a-7903a6881add None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:55.597911 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 399/820] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:55 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 31 14:25:55.599351 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befead29-fd7a-442c-ba6c-922226de3616 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:55.599642 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befead29-fd7a-442c-ba6c-922226de3616 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:55.599915 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befead29-fd7a-442c-ba6c-922226de3616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:55.617246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-befead29-fd7a-442c-ba6c-922226de3616 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:55.618231 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb19fd6a-227a-4366-8e4b-dbdf5ec00a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:55.618708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-befead29-fd7a-442c-ba6c-922226de3616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:55.618970 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19fd6a-227a-4366-8e4b-dbdf5ec00a74 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:55.619487 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19fd6a-227a-4366-8e4b-dbdf5ec00a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:55.619602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19fd6a-227a-4366-8e4b-dbdf5ec00a74 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:55.620804 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb19fd6a-227a-4366-8e4b-dbdf5ec00a74 None admin] RBAC: Adding query filter params (type=network) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:55.621116 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb19fd6a-227a-4366-8e4b-dbdf5ec00a74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:55.622737 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb19fd6a-227a-4366-8e4b-dbdf5ec00a74 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:55.629728 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 422/821] 10.222.0.67 () {56 vars in 1112 bytes} [Tue May 31 14:25:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:55.630965 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb19fd6a-227a-4366-8e4b-dbdf5ec00a74 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:55.670273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:55.670464 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:55.702779 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 400/822] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:25:55.703558 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befead29-fd7a-442c-ba6c-922226de3616 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:55.754966 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb19fd6a-227a-4366-8e4b-dbdf5ec00a74 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:55.802499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4384a5-13fa-4abf-a183-a7ca8b9e779b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:55.803796 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4384a5-13fa-4abf-a183-a7ca8b9e779b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:55.804245 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4384a5-13fa-4abf-a183-a7ca8b9e779b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:55.804683 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4384a5-13fa-4abf-a183-a7ca8b9e779b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:55.819794 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4384a5-13fa-4abf-a183-a7ca8b9e779b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:55.821367 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4384a5-13fa-4abf-a183-a7ca8b9e779b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:55.829788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befead29-fd7a-442c-ba6c-922226de3616 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:55.840230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-1c4384a5-13fa-4abf-a183-a7ca8b9e779b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:55.840230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:55.840230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:55.840230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:55.840230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:55.840230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:55.840230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:55.840230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:55.840230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:55.840230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:55.840230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:55.840230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:55.840230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:55.840230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:55.840230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:55.841150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:55.841150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:55.841150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:55.841150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:55.841150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:55.841150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:55.841150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:55.841150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:55.841150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:55.841150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:55.841150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:55.841150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:55.841150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:55.841150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:55.841150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:55.841150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:55.841150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:55.842043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:55.842043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:55.842043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:55.842043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:55.842285 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 423/823] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:55.842994 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4384a5-13fa-4abf-a183-a7ca8b9e779b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:55.880740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d288b96-2281-4097-884c-e74068855608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:55.881752 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d288b96-2281-4097-884c-e74068855608 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:55.882253 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d288b96-2281-4097-884c-e74068855608 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:55.882765 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d288b96-2281-4097-884c-e74068855608 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:55.883705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d288b96-2281-4097-884c-e74068855608 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:55.885962 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d288b96-2281-4097-884c-e74068855608 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:55.916259 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 401/824] 10.222.0.67 () {60 vars in 1144 bytes} [Tue May 31 14:25:55 2022] POST /identity/v3/endpoints => generated 327 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:25:55.918250 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d288b96-2281-4097-884c-e74068855608 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:55.934865 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:55.963479 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4384a5-13fa-4abf-a183-a7ca8b9e779b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:55.976643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:56.010519 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae8d5cf-29d6-4627-a37e-adc07115ddf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:56.011402 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8d5cf-29d6-4627-a37e-adc07115ddf1 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:56.011685 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8d5cf-29d6-4627-a37e-adc07115ddf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:56.011962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8d5cf-29d6-4627-a37e-adc07115ddf1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:56.012978 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae8d5cf-29d6-4627-a37e-adc07115ddf1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:56.014169 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae8d5cf-29d6-4627-a37e-adc07115ddf1 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:56.027702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-eae8d5cf-29d6-4627-a37e-adc07115ddf1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 31 14:25:56.027702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:56.027702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:25:56.027702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:25:56.027702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:25:56.027702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:25:56.027702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:25:56.027702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:25:56.027702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 31 14:25:56.027702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:56.027702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:25:56.027702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:56.027702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:56.027702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:56.027702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:56.027702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:56.028733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:56.028733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:56.028733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:56.028733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:56.028733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:56.028733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:56.028733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:56.028733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:25:56.028733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:25:56.028733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:25:56.028733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:25:56.028733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:56.028733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:56.028733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:56.028733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:56.028733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:56.029572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:56.029572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:56.029572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:56.029572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:56.029572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:56.029572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:56.029572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:56.029572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:56.029572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:56.029572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:56.029572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:56.029572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:25:56.029572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:25:56.029572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 31 14:25:56.029572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:56.029572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 424/825] 10.222.0.67 () {56 vars in 1129 bytes} [Tue May 31 14:25:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:56.035930 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae8d5cf-29d6-4627-a37e-adc07115ddf1 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:56.095385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d288b96-2281-4097-884c-e74068855608 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:56.191093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec026e5-a85c-4243-aa5f-12dbd3e96c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:56.191696 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec026e5-a85c-4243-aa5f-12dbd3e96c89 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:56.191984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec026e5-a85c-4243-aa5f-12dbd3e96c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:56.192261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec026e5-a85c-4243-aa5f-12dbd3e96c89 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:56.193102 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec026e5-a85c-4243-aa5f-12dbd3e96c89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:56.193466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec026e5-a85c-4243-aa5f-12dbd3e96c89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:56.194992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec026e5-a85c-4243-aa5f-12dbd3e96c89 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:56.208425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 402/826] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:56.213931 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec026e5-a85c-4243-aa5f-12dbd3e96c89 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:56.237512 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae8d5cf-29d6-4627-a37e-adc07115ddf1 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:56.272107 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0a4e0-9066-46d4-bd14-9559f623e4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:56.272780 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0a4e0-9066-46d4-bd14-9559f623e4ba None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:56.273074 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0a4e0-9066-46d4-bd14-9559f623e4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:56.273355 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0a4e0-9066-46d4-bd14-9559f623e4ba None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:56.274066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0a4e0-9066-46d4-bd14-9559f623e4ba None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:56.274380 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0a4e0-9066-46d4-bd14-9559f623e4ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:56.275802 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0a4e0-9066-46d4-bd14-9559f623e4ba None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:56.285732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 425/827] 10.222.0.67 () {56 vars in 1124 bytes} [Tue May 31 14:25:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:56.288131 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0a4e0-9066-46d4-bd14-9559f623e4ba None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:56.300043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:56.323456 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:56.334428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec026e5-a85c-4243-aa5f-12dbd3e96c89 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:56.376592 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fda88-20e4-47e6-b048-0f2a3295a58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:56.377964 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fda88-20e4-47e6-b048-0f2a3295a58f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:56.379243 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fda88-20e4-47e6-b048-0f2a3295a58f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:56.379746 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fda88-20e4-47e6-b048-0f2a3295a58f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:56.388365 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543fda88-20e4-47e6-b048-0f2a3295a58f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:56.390699 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543fda88-20e4-47e6-b048-0f2a3295a58f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:56.409926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-543fda88-20e4-47e6-b048-0f2a3295a58f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:56.409926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:56.409926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:56.409926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:56.409926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:56.409926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:56.409926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:56.409926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:56.409926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:56.409926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:56.409926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:56.409926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:56.409926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:25:56.409926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:25:56.409926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:25:56.410950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:25:56.410950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:56.410950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:56.410950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:56.410950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:56.410950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:56.410950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:56.410950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:56.410950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:56.410950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:56.410950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:56.410950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:56.410950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:56.410950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:56.410950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:56.410950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:56.410950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:56.411855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:25:56.411855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:25:56.411855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:25:56.411855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:25:56.411855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:25:56.411855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:25:56.411855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:56.411855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:56.412305 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 403/828] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:25:56.418550 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fda88-20e4-47e6-b048-0f2a3295a58f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:56.421108 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0a4e0-9066-46d4-bd14-9559f623e4ba None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:56.471097 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a548c0-ab7d-43bb-85a3-7e8a67de54d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:56.472379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a548c0-ab7d-43bb-85a3-7e8a67de54d4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:56.472881 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a548c0-ab7d-43bb-85a3-7e8a67de54d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:56.473455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a548c0-ab7d-43bb-85a3-7e8a67de54d4 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:56.474470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a548c0-ab7d-43bb-85a3-7e8a67de54d4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:56.475067 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a548c0-ab7d-43bb-85a3-7e8a67de54d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:56.476901 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a548c0-ab7d-43bb-85a3-7e8a67de54d4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:56.483951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 426/829] 10.222.0.67 () {56 vars in 1124 bytes} [Tue May 31 14:25:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:25:56.491124 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a548c0-ab7d-43bb-85a3-7e8a67de54d4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:56.543975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fda88-20e4-47e6-b048-0f2a3295a58f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:56.571161 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:56.591482 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc5e15-1012-47d9-8e25-7a7aea40a185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:56.592786 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc5e15-1012-47d9-8e25-7a7aea40a185 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:56.593275 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc5e15-1012-47d9-8e25-7a7aea40a185 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:56.593807 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc5e15-1012-47d9-8e25-7a7aea40a185 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:56.595326 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc5e15-1012-47d9-8e25-7a7aea40a185 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:56.595963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc5e15-1012-47d9-8e25-7a7aea40a185 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:56.597961 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc5e15-1012-47d9-8e25-7a7aea40a185 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:56.607212 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a548c0-ab7d-43bb-85a3-7e8a67de54d4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:56.615001 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 404/830] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:25:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:56.622031 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc5e15-1012-47d9-8e25-7a7aea40a185 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:56.642259 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:56.654611 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5308604-e5e2-4164-b124-d81f462d7b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:56.655700 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5308604-e5e2-4164-b124-d81f462d7b0b None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:56.656088 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5308604-e5e2-4164-b124-d81f462d7b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:56.656796 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5308604-e5e2-4164-b124-d81f462d7b0b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:56.657933 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5308604-e5e2-4164-b124-d81f462d7b0b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:56.659602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5308604-e5e2-4164-b124-d81f462d7b0b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:56.686806 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 427/831] 10.222.0.67 () {60 vars in 1144 bytes} [Tue May 31 14:25:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:25:56.747925 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc5e15-1012-47d9-8e25-7a7aea40a185 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:56.800691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86c302-8645-4628-8d61-3eb2efd2d81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:56.801685 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86c302-8645-4628-8d61-3eb2efd2d81d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:56.802120 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86c302-8645-4628-8d61-3eb2efd2d81d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:56.802557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86c302-8645-4628-8d61-3eb2efd2d81d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:56.803630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86c302-8645-4628-8d61-3eb2efd2d81d None admin] RBAC: Adding query filter params (role.id=e8d679596ffb45939229d1e2a13ce9b0, scope.project.id=f965a93a2f674598890bd8cafededb2c, user.id=68c6363b60954c3ebf50d2019f5f2fd2) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:56.804107 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86c302-8645-4628-8d61-3eb2efd2d81d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:56.805725 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86c302-8645-4628-8d61-3eb2efd2d81d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:56.813331 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 405/832] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:25:56 2022] GET /identity/v3/role_assignments?user.id=68c6363b60954c3ebf50d2019f5f2fd2&scope.project.id=f965a93a2f674598890bd8cafededb2c&role.id=e8d679596ffb45939229d1e2a13ce9b0 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:57.557700 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d0a08-9131-4dcb-8004-5d958ac25e96 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:57.557884 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d0a08-9131-4dcb-8004-5d958ac25e96 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:57.558190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d0a08-9131-4dcb-8004-5d958ac25e96 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:57.559929 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 428/833] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:57.565909 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352360b-923a-49ab-b963-36a18afc7dfe None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:57.566446 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352360b-923a-49ab-b963-36a18afc7dfe None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:57.566446 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352360b-923a-49ab-b963-36a18afc7dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:57.584181 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e352360b-923a-49ab-b963-36a18afc7dfe None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:57.585360 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e352360b-923a-49ab-b963-36a18afc7dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:57.653056 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 406/834] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:57 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:25:57.676641 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d0a08-9131-4dcb-8004-5d958ac25e96 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:57.785434 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d0a08-9131-4dcb-8004-5d958ac25e96 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:57.812126 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:57.821428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c919cf17-9349-421a-bd27-c5fd513021a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:57.822579 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919cf17-9349-421a-bd27-c5fd513021a5 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:57.823534 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919cf17-9349-421a-bd27-c5fd513021a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:57.823815 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919cf17-9349-421a-bd27-c5fd513021a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:57.835753 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c919cf17-9349-421a-bd27-c5fd513021a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:57.843475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-c919cf17-9349-421a-bd27-c5fd513021a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:57.843475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:57.843475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:57.843475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:57.843475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:57.843475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:57.843475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:57.843475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:57.843475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:57.843475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:57.843475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:57.843475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:57.843475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:57.843475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:57.843475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:57.843475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:57.844467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:57.844467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:57.844467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:57.844467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:57.844467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:57.844467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:57.844467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:57.844467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:57.844467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:57.844467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:57.844467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:57.844467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:57.844467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:57.844467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:57.844467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:57.844467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:57.845306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:57.845306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:57.845306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:57.845306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:57.845306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:57.845306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:57.845306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:57.845306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:57.845306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:57.845306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:57.845306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 31 14:25:57.845306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:57.845306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 429/835] 10.222.0.67 () {56 vars in 1102 bytes} [Tue May 31 14:25:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:57.851161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352360b-923a-49ab-b963-36a18afc7dfe None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:57.871023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:57.885974 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5d966-0f6c-4e4f-aa23-383128ee573a None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:57.886274 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5d966-0f6c-4e4f-aa23-383128ee573a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:57.886526 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5d966-0f6c-4e4f-aa23-383128ee573a None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:57.888054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 430/836] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:57.893206 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590132ca-c349-4811-b37c-175c819fa039 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:57.893737 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590132ca-c349-4811-b37c-175c819fa039 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:57.894218 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590132ca-c349-4811-b37c-175c819fa039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:57.915086 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-590132ca-c349-4811-b37c-175c819fa039 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:57.916285 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-590132ca-c349-4811-b37c-175c819fa039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:57.967470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352360b-923a-49ab-b963-36a18afc7dfe None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:58.004337 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca11a8-7090-49f2-abf1-903721ee254e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:58.005015 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca11a8-7090-49f2-abf1-903721ee254e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:58.005301 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca11a8-7090-49f2-abf1-903721ee254e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:58.005576 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca11a8-7090-49f2-abf1-903721ee254e None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:58.006321 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca11a8-7090-49f2-abf1-903721ee254e None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:58.006581 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 431/837] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:57 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 31 14:25:58.006717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca11a8-7090-49f2-abf1-903721ee254e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:58.008009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca11a8-7090-49f2-abf1-903721ee254e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:58.017257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 407/838] 10.222.0.67 () {56 vars in 1103 bytes} [Tue May 31 14:25:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:58.023016 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590132ca-c349-4811-b37c-175c819fa039 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:58.030398 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca11a8-7090-49f2-abf1-903721ee254e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:58.103529 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:58.132951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590132ca-c349-4811-b37c-175c819fa039 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:58.151910 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:58.154059 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca11a8-7090-49f2-abf1-903721ee254e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:58.171013 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bd313-264d-4814-aa7b-4c6488661fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:58.171967 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bd313-264d-4814-aa7b-4c6488661fff None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:58.172252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bd313-264d-4814-aa7b-4c6488661fff None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:58.172528 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bd313-264d-4814-aa7b-4c6488661fff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:58.183812 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bd313-264d-4814-aa7b-4c6488661fff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:58.185388 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bd313-264d-4814-aa7b-4c6488661fff None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:58.191163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f4b88-fc84-4956-862f-ccb65dd6862e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:58.192232 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f4b88-fc84-4956-862f-ccb65dd6862e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:58.192692 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f4b88-fc84-4956-862f-ccb65dd6862e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:58.193262 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f4b88-fc84-4956-862f-ccb65dd6862e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:58.204863 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f4b88-fc84-4956-862f-ccb65dd6862e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:25:58.205883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-395bd313-264d-4814-aa7b-4c6488661fff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:58.205883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:58.205883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:58.205883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:58.205883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:58.205883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:58.205883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:58.205883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:58.205883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:58.205883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:58.205883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:58.205883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:58.205883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:58.205883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:58.205883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:58.206791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:58.206791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:58.206791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:58.206791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:58.206791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:58.206791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:58.206791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:58.206791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:58.206791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:58.206791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:58.206791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:58.206791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:58.206791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:58.206791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:58.206791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:58.206791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:58.206791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:58.207671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:58.207671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:58.207671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:58.207671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:58.211866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 432/839] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:58.213089 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bd313-264d-4814-aa7b-4c6488661fff None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:58.216414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-6e2f4b88-fc84-4956-862f-ccb65dd6862e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:25:58.216414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:58.216414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:58.216414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:58.216414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:58.216414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:58.216414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:58.216414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:58.216414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:58.216414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:58.216414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:58.216414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:58.216414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:25:58.216414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:25:58.216414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:25:58.216414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:25:58.217317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:25:58.217317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:25:58.217317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:25:58.217317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:25:58.217317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:58.217317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:58.217317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:58.217317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:58.217317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:58.217317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:58.217317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:58.217317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:58.217317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:58.217317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:58.217317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:58.217317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:58.218139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:58.218139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:58.218139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:58.218139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:58.218139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:25:58.218139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:25:58.218139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:25:58.218139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:25:58.218139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:25:58.218139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:25:58.218139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:25:58.218139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:58.220840 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 408/840] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:25:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:25:58.228755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f4b88-fc84-4956-862f-ccb65dd6862e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:58.320069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bd313-264d-4814-aa7b-4c6488661fff None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:58.334882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f4b88-fc84-4956-862f-ccb65dd6862e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:58.358511 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22c530-4fff-49c6-a67d-01ff69f867d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:58.359403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22c530-4fff-49c6-a67d-01ff69f867d3 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:58.359699 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22c530-4fff-49c6-a67d-01ff69f867d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:58.359981 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22c530-4fff-49c6-a67d-01ff69f867d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:58.360817 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22c530-4fff-49c6-a67d-01ff69f867d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:58.361136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22c530-4fff-49c6-a67d-01ff69f867d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:58.362519 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22c530-4fff-49c6-a67d-01ff69f867d3 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:58.363754 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:58.372867 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5ed6d-8ddb-404a-a993-1cf0e6e03b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:58.373620 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5ed6d-8ddb-404a-a993-1cf0e6e03b1f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:58.373972 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5ed6d-8ddb-404a-a993-1cf0e6e03b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:58.374422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5ed6d-8ddb-404a-a993-1cf0e6e03b1f None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:58.375291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5ed6d-8ddb-404a-a993-1cf0e6e03b1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:58.375673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5ed6d-8ddb-404a-a993-1cf0e6e03b1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:58.377102 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5ed6d-8ddb-404a-a993-1cf0e6e03b1f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:58.382324 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 433/841] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:58.383655 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22c530-4fff-49c6-a67d-01ff69f867d3 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:58.390920 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 409/842] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:25:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:58.391782 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5ed6d-8ddb-404a-a993-1cf0e6e03b1f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:58.413726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:58.495115 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22c530-4fff-49c6-a67d-01ff69f867d3 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:58.500541 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5ed6d-8ddb-404a-a993-1cf0e6e03b1f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:58.535462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e598dac-d465-4af4-bf56-53925bf57a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:58.536476 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e598dac-d465-4af4-bf56-53925bf57a6f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:58.536855 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e598dac-d465-4af4-bf56-53925bf57a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:58.537239 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e598dac-d465-4af4-bf56-53925bf57a6f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:58.537636 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4a596-3ce0-4d31-aab4-0b1ae05af2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:58.538617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4a596-3ce0-4d31-aab4-0b1ae05af2f7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:58.538948 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4a596-3ce0-4d31-aab4-0b1ae05af2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:58.539374 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4a596-3ce0-4d31-aab4-0b1ae05af2f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:58.545457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e598dac-d465-4af4-bf56-53925bf57a6f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:58.547040 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e598dac-d465-4af4-bf56-53925bf57a6f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:58.552455 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4a596-3ce0-4d31-aab4-0b1ae05af2f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:58.553943 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4a596-3ce0-4d31-aab4-0b1ae05af2f7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:58.565431 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-2e598dac-d465-4af4-bf56-53925bf57a6f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 31 14:25:58.565431 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:58.565431 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:58.565431 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:58.565431 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:58.565431 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:58.565431 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:58.565431 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:58.565431 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:58.565431 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:58.565431 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:58.565431 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:58.565431 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:25:58.565431 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:25:58.565431 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:25:58.565431 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:25:58.567606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:58.567606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:58.567606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:25:58.567606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:58.567606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:25:58.567606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:58.567606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:58.567606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:58.567606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:58.567606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:58.567606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:58.567606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:58.567606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:58.567606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:58.567606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:58.567606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:58.567606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:58.567606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:58.569800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:58.569800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:58.569800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:25:58.569800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:25:58.569800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:25:58.569800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:25:58.569800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:25:58.569800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:25:58.569800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 31 14:25:58.569800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:58.576136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 434/843] 10.222.0.67 () {56 vars in 1117 bytes} [Tue May 31 14:25:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:58.576136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e598dac-d465-4af4-bf56-53925bf57a6f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:58.576538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-49d4a596-3ce0-4d31-aab4-0b1ae05af2f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:58.576538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:58.576538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:58.576538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:58.576538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:58.576538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:58.576538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:58.576538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:58.576538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:58.576538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:58.576538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:58.576538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:58.576538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:58.576538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:58.576538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:58.578762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:58.578762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:58.578762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:58.578762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:58.578762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:58.578762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:58.578762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:58.578762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:58.578762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:58.578762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:58.578762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:58.578762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:58.578762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:58.578762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:58.578762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:58.578762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:58.578762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:58.580882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:58.580882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:58.580882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:58.580882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:58.580882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 410/844] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:25:58.580882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4a596-3ce0-4d31-aab4-0b1ae05af2f7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:58.627254 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:58.670529 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:58.684123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e598dac-d465-4af4-bf56-53925bf57a6f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:58.689577 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4a596-3ce0-4d31-aab4-0b1ae05af2f7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:58.726575 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11d046-90b7-46b2-bcb1-fe9693d2dc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:58.727762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11d046-90b7-46b2-bcb1-fe9693d2dc36 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:58.728292 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11d046-90b7-46b2-bcb1-fe9693d2dc36 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:58.728721 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11d046-90b7-46b2-bcb1-fe9693d2dc36 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:58.730382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa11d046-90b7-46b2-bcb1-fe9693d2dc36 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:58.730574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a74075-4591-4373-968e-8ab578597093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:58.731049 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa11d046-90b7-46b2-bcb1-fe9693d2dc36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:58.731279 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a74075-4591-4373-968e-8ab578597093 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:58.731565 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a74075-4591-4373-968e-8ab578597093 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:58.731843 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a74075-4591-4373-968e-8ab578597093 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:58.732570 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a74075-4591-4373-968e-8ab578597093 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:58.732883 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a74075-4591-4373-968e-8ab578597093 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:58.733494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa11d046-90b7-46b2-bcb1-fe9693d2dc36 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:58.734246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a74075-4591-4373-968e-8ab578597093 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:58.752502 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 411/845] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:58.753393 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a74075-4591-4373-968e-8ab578597093 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:58.758672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 435/846] 10.222.0.67 () {56 vars in 1149 bytes} [Tue May 31 14:25:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:25:58.760056 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11d046-90b7-46b2-bcb1-fe9693d2dc36 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:58.864082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a74075-4591-4373-968e-8ab578597093 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:58.866032 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:58.874310 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11d046-90b7-46b2-bcb1-fe9693d2dc36 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:58.903089 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750486f1-2e5e-4e6b-9666-ecc2964197a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:58.904175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750486f1-2e5e-4e6b-9666-ecc2964197a7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:58.904298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750486f1-2e5e-4e6b-9666-ecc2964197a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:58.904582 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750486f1-2e5e-4e6b-9666-ecc2964197a7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:58.912918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750486f1-2e5e-4e6b-9666-ecc2964197a7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:58.913178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a45be1-2c39-42c6-97cc-d64f87725abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:58.914072 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a45be1-2c39-42c6-97cc-d64f87725abf None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:58.914372 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750486f1-2e5e-4e6b-9666-ecc2964197a7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:58.914713 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a45be1-2c39-42c6-97cc-d64f87725abf None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:58.915292 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a45be1-2c39-42c6-97cc-d64f87725abf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:58.931023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-750486f1-2e5e-4e6b-9666-ecc2964197a7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 31 14:25:58.931023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:58.931023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:58.931023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:58.931023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:58.931023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:58.931023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:58.931023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:58.931023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:58.931023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:58.931023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:58.931023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:58.931023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:25:58.931023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:25:58.931023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:25:58.931023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:25:58.932879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:58.932879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:58.932879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:25:58.932879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:58.932879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:25:58.932879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:25:58.932879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:58.932879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:58.932879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:58.932879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:58.932879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:58.932879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:58.932879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:58.932879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:58.932879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:58.932879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:58.932879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:58.932879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:58.934065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:58.934065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:58.934065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:25:58.934065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:25:58.934065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:25:58.934065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:25:58.934065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:25:58.934065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:25:58.934065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 31 14:25:58.934065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:58.939347 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:58.939643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 412/847] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:25:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:58.941231 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750486f1-2e5e-4e6b-9666-ecc2964197a7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:58.996619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a45be1-2c39-42c6-97cc-d64f87725abf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:58.998687 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a45be1-2c39-42c6-97cc-d64f87725abf None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:59.051657 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750486f1-2e5e-4e6b-9666-ecc2964197a7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:59.090531 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dce454-5f94-4f3e-a56a-3f4f0d99e543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:59.091240 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dce454-5f94-4f3e-a56a-3f4f0d99e543 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:59.091531 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dce454-5f94-4f3e-a56a-3f4f0d99e543 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:59.091815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dce454-5f94-4f3e-a56a-3f4f0d99e543 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:59.092657 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dce454-5f94-4f3e-a56a-3f4f0d99e543 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:59.092973 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dce454-5f94-4f3e-a56a-3f4f0d99e543 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:59.094418 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dce454-5f94-4f3e-a56a-3f4f0d99e543 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:59.108138 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 436/848] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:58 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 31 14:25:59.109558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f0aff-f5d3-4346-8656-89125e1d1ce2 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:59.109855 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f0aff-f5d3-4346-8656-89125e1d1ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:59.110137 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f0aff-f5d3-4346-8656-89125e1d1ce2 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:59.113474 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 437/849] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:59.113474 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f0aff-f5d3-4346-8656-89125e1d1ce2 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:59.118504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 413/850] 10.222.0.67 () {56 vars in 1135 bytes} [Tue May 31 14:25:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:59.120087 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d4324-850a-4ff1-b77b-03137d95478e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:59.120522 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d4324-850a-4ff1-b77b-03137d95478e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:59.120936 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d4324-850a-4ff1-b77b-03137d95478e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:59.144371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-136d4324-850a-4ff1-b77b-03137d95478e None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:59.145765 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-136d4324-850a-4ff1-b77b-03137d95478e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:59.162770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:59.168816 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:59.220681 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 414/851] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:25:59.221793 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d4324-850a-4ff1-b77b-03137d95478e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:59.230403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f0aff-f5d3-4346-8656-89125e1d1ce2 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:59.265209 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497bf2d3-27cc-48e5-bedd-12668e6084e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:59.266002 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bf2d3-27cc-48e5-bedd-12668e6084e1 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:59.266284 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bf2d3-27cc-48e5-bedd-12668e6084e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:59.266582 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bf2d3-27cc-48e5-bedd-12668e6084e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:59.276520 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497bf2d3-27cc-48e5-bedd-12668e6084e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:59.277970 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497bf2d3-27cc-48e5-bedd-12668e6084e1 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:59.295294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-497bf2d3-27cc-48e5-bedd-12668e6084e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:59.295294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:59.295294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:59.295294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:59.295294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:59.295294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:59.295294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:59.295294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:59.295294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:59.295294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:59.295294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:59.295294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:59.295294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:59.295294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:59.295294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:59.296420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:59.296420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:59.296420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:59.296420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:59.296420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:59.296420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:59.296420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:59.296420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:59.296420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:59.296420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:59.296420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:59.296420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:59.296420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:59.296420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:59.296420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:59.296420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:59.296420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:59.297577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:59.297577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:59.297577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:59.297577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:59.297577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 438/852] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:25:59.297960 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e0137-2491-406e-93f4-76010fea3ce1 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:59.298141 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e0137-2491-406e-93f4-76010fea3ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:59.298420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e0137-2491-406e-93f4-76010fea3ce1 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:59.299837 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 439/853] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:25:59.300410 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e0137-2491-406e-93f4-76010fea3ce1 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:59.327779 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d4324-850a-4ff1-b77b-03137d95478e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:59.361017 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a5f3f-62ef-411d-ab37-b6da304e377f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:59.361666 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a5f3f-62ef-411d-ab37-b6da304e377f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:59.361953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a5f3f-62ef-411d-ab37-b6da304e377f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:59.362236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a5f3f-62ef-411d-ab37-b6da304e377f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:59.380493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:59.397107 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:59.402116 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e0137-2491-406e-93f4-76010fea3ce1 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:59.430528 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a5f3f-62ef-411d-ab37-b6da304e377f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:59.432484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a5f3f-62ef-411d-ab37-b6da304e377f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:59.435371 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4493f2f-1299-4ca1-84d3-2d94d29bc83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:59.436159 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4493f2f-1299-4ca1-84d3-2d94d29bc83e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:59.436444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4493f2f-1299-4ca1-84d3-2d94d29bc83e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:59.436721 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4493f2f-1299-4ca1-84d3-2d94d29bc83e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:59.437374 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4493f2f-1299-4ca1-84d3-2d94d29bc83e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:59.438785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4493f2f-1299-4ca1-84d3-2d94d29bc83e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:59.451289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-b4493f2f-1299-4ca1-84d3-2d94d29bc83e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 31 14:25:59.451289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:59.451289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:25:59.451289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:25:59.451289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:25:59.451289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:25:59.451289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:25:59.451289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:25:59.451289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 31 14:25:59.451289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:59.451289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:25:59.451289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:59.451289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:59.451289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:59.451289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:59.451289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:59.451289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:59.452354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:59.452354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:59.452354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:59.452354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:59.452354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:59.452354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:59.452354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:25:59.452354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:25:59.452354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:25:59.452354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:25:59.452354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:59.452354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:59.452354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:59.452354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:59.452354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:59.452354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:59.453236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:59.453236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:59.453236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:59.453236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:59.453236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:59.453236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:59.453236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:59.453236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:59.453236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:59.453236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:59.453236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:25:59.453236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:25:59.453236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 31 14:25:59.453236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:59.453236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 440/854] 10.222.0.67 () {56 vars in 1114 bytes} [Tue May 31 14:25:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:59.454064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4493f2f-1299-4ca1-84d3-2d94d29bc83e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:59.533877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 415/855] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:25:59 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 31 14:25:59.535203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce32939-5dfd-4bd5-8f7c-e33903f97dac None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:59.535504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce32939-5dfd-4bd5-8f7c-e33903f97dac None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:59.535788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce32939-5dfd-4bd5-8f7c-e33903f97dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:59.551900 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9ce32939-5dfd-4bd5-8f7c-e33903f97dac None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:25:59.553087 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9ce32939-5dfd-4bd5-8f7c-e33903f97dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:25:59.554911 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4493f2f-1299-4ca1-84d3-2d94d29bc83e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:59.588593 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06829f7-75f7-4ed2-9170-f93384a963af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:59.589248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06829f7-75f7-4ed2-9170-f93384a963af None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:59.589536 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06829f7-75f7-4ed2-9170-f93384a963af None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:59.589814 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06829f7-75f7-4ed2-9170-f93384a963af None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:59.590534 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06829f7-75f7-4ed2-9170-f93384a963af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:59.590879 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06829f7-75f7-4ed2-9170-f93384a963af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:59.592253 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06829f7-75f7-4ed2-9170-f93384a963af None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:59.598084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:59.604631 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 441/856] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:59.605561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06829f7-75f7-4ed2-9170-f93384a963af None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:59.620492 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 416/857] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:25:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:25:59.621314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce32939-5dfd-4bd5-8f7c-e33903f97dac None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:59.638996 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:59.713575 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06829f7-75f7-4ed2-9170-f93384a963af None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:59.719414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce32939-5dfd-4bd5-8f7c-e33903f97dac None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:59.746915 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f97be-ca6d-40a3-afa5-27c23469d9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:59.747568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f97be-ca6d-40a3-afa5-27c23469d9ca None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:59.747854 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f97be-ca6d-40a3-afa5-27c23469d9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:59.748129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f97be-ca6d-40a3-afa5-27c23469d9ca None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:59.748829 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f97be-ca6d-40a3-afa5-27c23469d9ca None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:25:59.749131 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f97be-ca6d-40a3-afa5-27c23469d9ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:59.750487 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f97be-ca6d-40a3-afa5-27c23469d9ca None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:59.753461 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4dd069-7cbc-4598-b422-35db7923b5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:59.754250 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4dd069-7cbc-4598-b422-35db7923b5ff None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:59.754537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4dd069-7cbc-4598-b422-35db7923b5ff None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:59.754847 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4dd069-7cbc-4598-b422-35db7923b5ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:59.754983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 442/858] 10.222.0.67 () {56 vars in 1114 bytes} [Tue May 31 14:25:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:25:59.759806 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f97be-ca6d-40a3-afa5-27c23469d9ca None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:59.765492 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4dd069-7cbc-4598-b422-35db7923b5ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:59.766953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4dd069-7cbc-4598-b422-35db7923b5ff None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:59.784114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-7f4dd069-7cbc-4598-b422-35db7923b5ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:59.784114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:59.784114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:59.784114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:59.784114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:59.784114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:59.784114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:59.784114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:59.784114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:59.784114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:59.784114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:59.784114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:59.784114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:25:59.784114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:25:59.784114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:25:59.785187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:25:59.785187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:59.785187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:59.785187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:59.785187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:59.785187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:59.785187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:59.785187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:59.785187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:59.785187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:59.785187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:59.785187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:59.785187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:59.785187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:59.785187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:59.785187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:59.785187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:59.786371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:25:59.786371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:25:59.786371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:25:59.786371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:59.786371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 417/859] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:25:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:59.786371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4dd069-7cbc-4598-b422-35db7923b5ff None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:59.817063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:59.844058 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:25:59.859800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f97be-ca6d-40a3-afa5-27c23469d9ca None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:59.889934 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4dd069-7cbc-4598-b422-35db7923b5ff None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:25:59.892305 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd58af-9263-42b2-bd24-053e8bf35d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:59.893101 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd58af-9263-42b2-bd24-053e8bf35d5f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:59.893381 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd58af-9263-42b2-bd24-053e8bf35d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:59.893659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd58af-9263-42b2-bd24-053e8bf35d5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:59.901958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd58af-9263-42b2-bd24-053e8bf35d5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:59.903608 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd58af-9263-42b2-bd24-053e8bf35d5f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:59.917496 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-acbd58af-9263-42b2-bd24-053e8bf35d5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:59.917496 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:59.917496 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:59.917496 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:59.917496 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:59.917496 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:59.917496 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:59.917496 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:59.917496 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:59.917496 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:59.917496 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:59.917496 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:59.917496 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:25:59.917496 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:25:59.917496 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:25:59.918580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:25:59.918580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:59.918580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:59.918580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:59.918580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:59.918580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:59.918580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:59.918580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:59.918580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:25:59.918580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:59.918580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:59.918580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:59.918580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:59.918580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:59.918580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:59.918580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:59.918580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:59.919746 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:25:59.919746 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:25:59.919746 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:25:59.919746 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:25:59.919746 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:25:59.919746 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:25:59.919746 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:25:59.919746 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:25:59.919746 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 443/860] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:25:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:59.919746 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd58af-9263-42b2-bd24-053e8bf35d5f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:25:59.928022 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d8e1d-baaa-4bb6-92ac-976d136b4b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:25:59.928842 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d8e1d-baaa-4bb6-92ac-976d136b4b53 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:25:59.929124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d8e1d-baaa-4bb6-92ac-976d136b4b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:25:59.929399 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d8e1d-baaa-4bb6-92ac-976d136b4b53 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:25:59.930061 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d8e1d-baaa-4bb6-92ac-976d136b4b53 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:25:59.931469 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d8e1d-baaa-4bb6-92ac-976d136b4b53 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:25:59.943457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-966d8e1d-baaa-4bb6-92ac-976d136b4b53 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 31 14:25:59.943457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:59.943457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:25:59.943457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:25:59.943457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:25:59.943457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:25:59.943457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:25:59.943457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:25:59.943457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 31 14:25:59.943457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:59.943457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:25:59.943457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:59.943457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:25:59.943457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:25:59.943457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:25:59.943457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:25:59.944532 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:25:59.944532 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:25:59.944532 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:25:59.944532 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:25:59.944532 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:25:59.944532 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:25:59.944532 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:25:59.944532 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:25:59.944532 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:25:59.944532 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:25:59.944532 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:25:59.944532 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:25:59.944532 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:25:59.944532 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:25:59.944532 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:25:59.944532 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:25:59.945381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:25:59.945381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:25:59.945381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:25:59.945381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:25:59.945381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:25:59.945381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:25:59.945381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:25:59.945381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:25:59.945381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:25:59.945381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:25:59.945381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:25:59.945381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:25:59.945381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:25:59.945381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 31 14:25:59.945381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:25:59.945381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 418/861] 10.222.0.67 () {56 vars in 1132 bytes} [Tue May 31 14:25:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:25:59.946243 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d8e1d-baaa-4bb6-92ac-976d136b4b53 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:00.023675 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd58af-9263-42b2-bd24-053e8bf35d5f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:00.039216 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:00.046064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d8e1d-baaa-4bb6-92ac-976d136b4b53 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:00.057288 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68f110-8adc-41a5-a28d-401a0cd7aaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:00.057947 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68f110-8adc-41a5-a28d-401a0cd7aaf6 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:00.058230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68f110-8adc-41a5-a28d-401a0cd7aaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:00.058511 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68f110-8adc-41a5-a28d-401a0cd7aaf6 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:00.059253 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68f110-8adc-41a5-a28d-401a0cd7aaf6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:00.059566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68f110-8adc-41a5-a28d-401a0cd7aaf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:00.060933 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68f110-8adc-41a5-a28d-401a0cd7aaf6 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:00.065613 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 444/862] 10.222.0.67 () {56 vars in 1114 bytes} [Tue May 31 14:25:59 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:00.066377 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68f110-8adc-41a5-a28d-401a0cd7aaf6 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:00.082446 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6b013-5494-4f85-bba9-ae417baca30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:00.083205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6b013-5494-4f85-bba9-ae417baca30d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:00.083499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6b013-5494-4f85-bba9-ae417baca30d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:00.083780 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6b013-5494-4f85-bba9-ae417baca30d None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:00.084533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6b013-5494-4f85-bba9-ae417baca30d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:00.084852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6b013-5494-4f85-bba9-ae417baca30d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:00.086302 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6b013-5494-4f85-bba9-ae417baca30d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:00.086999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:00.100981 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 419/863] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:00.101656 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6b013-5494-4f85-bba9-ae417baca30d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:00.208942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68f110-8adc-41a5-a28d-401a0cd7aaf6 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:00.229744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6b013-5494-4f85-bba9-ae417baca30d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:00.247362 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc28a48-b5a6-4297-8df4-c9b5d5add3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:00.248044 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc28a48-b5a6-4297-8df4-c9b5d5add3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:00.248328 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc28a48-b5a6-4297-8df4-c9b5d5add3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:00.248602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc28a48-b5a6-4297-8df4-c9b5d5add3a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:00.249409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc28a48-b5a6-4297-8df4-c9b5d5add3a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:00.249720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc28a48-b5a6-4297-8df4-c9b5d5add3a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:00.251231 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc28a48-b5a6-4297-8df4-c9b5d5add3a9 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:00.265791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 445/864] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:26:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:00.267227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc28a48-b5a6-4297-8df4-c9b5d5add3a9 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:00.272580 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e78157-34ae-43d1-98a3-57176f0d2807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:00.273255 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e78157-34ae-43d1-98a3-57176f0d2807 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:00.273538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e78157-34ae-43d1-98a3-57176f0d2807 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:00.273864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e78157-34ae-43d1-98a3-57176f0d2807 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:00.274526 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e78157-34ae-43d1-98a3-57176f0d2807 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:00.274862 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e78157-34ae-43d1-98a3-57176f0d2807 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:00.276797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e78157-34ae-43d1-98a3-57176f0d2807 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:00.284052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 420/865] 10.222.0.67 () {56 vars in 1126 bytes} [Tue May 31 14:26:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:00.284850 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e78157-34ae-43d1-98a3-57176f0d2807 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:00.307579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:00.346142 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:00.378888 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc28a48-b5a6-4297-8df4-c9b5d5add3a9 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:00.394325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e78157-34ae-43d1-98a3-57176f0d2807 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:00.413946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e34815-0ced-4fb8-9319-59bffe5eaa46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:00.414844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e34815-0ced-4fb8-9319-59bffe5eaa46 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:00.415129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e34815-0ced-4fb8-9319-59bffe5eaa46 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:00.415405 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e34815-0ced-4fb8-9319-59bffe5eaa46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:00.422458 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e34815-0ced-4fb8-9319-59bffe5eaa46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:00.424191 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e34815-0ced-4fb8-9319-59bffe5eaa46 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:00.430116 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540fb15-5dbe-448f-9273-5132741664ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:00.430786 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540fb15-5dbe-448f-9273-5132741664ad None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:00.431074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540fb15-5dbe-448f-9273-5132741664ad None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:00.431353 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540fb15-5dbe-448f-9273-5132741664ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:00.432218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540fb15-5dbe-448f-9273-5132741664ad None admin] RBAC: Adding query filter params (role.id=5956aa478bfc4d03919395e637e61c83, scope.project.id=f965a93a2f674598890bd8cafededb2c, user.id=0b7d100f2407432381d119c25814fff3) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:00.432573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540fb15-5dbe-448f-9273-5132741664ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:00.433983 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540fb15-5dbe-448f-9273-5132741664ad None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:00.438077 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-a4e34815-0ced-4fb8-9319-59bffe5eaa46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:26:00.438077 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:00.438077 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:00.438077 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:00.438077 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:00.438077 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:00.438077 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:00.438077 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:00.438077 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:00.438077 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:00.438077 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:00.438077 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:00.438077 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:26:00.438077 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:26:00.438077 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:26:00.438977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:26:00.438977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:00.438977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:00.438977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:00.438977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:00.438977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:00.438977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:00.438977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:00.438977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:26:00.438977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:00.438977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:00.438977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:00.438977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:00.438977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:00.438977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:00.438977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:00.438977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:00.439891 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:26:00.439891 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:26:00.439891 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:26:00.439891 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:26:00.439891 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:26:00.439891 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:26:00.439891 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:26:00.439891 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:00.451019 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 421/866] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:26:00 2022] GET /identity/v3/role_assignments?user.id=0b7d100f2407432381d119c25814fff3&scope.project.id=f965a93a2f674598890bd8cafededb2c&role.id=5956aa478bfc4d03919395e637e61c83 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:00.451249 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 446/867] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:26:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:00.451351 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540fb15-5dbe-448f-9273-5132741664ad None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:00.452318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e34815-0ced-4fb8-9319-59bffe5eaa46 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:00.560054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:00.573275 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e34815-0ced-4fb8-9319-59bffe5eaa46 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:00.579940 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540fb15-5dbe-448f-9273-5132741664ad None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:00.606462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:00.613484 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087dc6a2-cec8-4909-9df1-180e8d365bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:00.614321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087dc6a2-cec8-4909-9df1-180e8d365bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:00.614738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087dc6a2-cec8-4909-9df1-180e8d365bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:00.615115 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087dc6a2-cec8-4909-9df1-180e8d365bf6 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:00.616200 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087dc6a2-cec8-4909-9df1-180e8d365bf6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:00.616577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087dc6a2-cec8-4909-9df1-180e8d365bf6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:00.618145 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087dc6a2-cec8-4909-9df1-180e8d365bf6 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:00.621993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51660a37-62f9-4775-acf4-b1162daa4bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:00.622844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51660a37-62f9-4775-acf4-b1162daa4bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:00.623200 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51660a37-62f9-4775-acf4-b1162daa4bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:00.623547 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51660a37-62f9-4775-acf4-b1162daa4bf1 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:00.624342 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51660a37-62f9-4775-acf4-b1162daa4bf1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:00.624725 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51660a37-62f9-4775-acf4-b1162daa4bf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:00.626184 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51660a37-62f9-4775-acf4-b1162daa4bf1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:00.631929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 422/868] 10.222.0.67 () {56 vars in 1126 bytes} [Tue May 31 14:26:00 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:00.633218 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 447/869] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:26:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:00.644145 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51660a37-62f9-4775-acf4-b1162daa4bf1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:00.761969 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51660a37-62f9-4775-acf4-b1162daa4bf1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:00.799952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375d1ac-e856-42a9-a071-8b01d6e95204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:00.800634 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375d1ac-e856-42a9-a071-8b01d6e95204 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:00.800919 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375d1ac-e856-42a9-a071-8b01d6e95204 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:00.801199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375d1ac-e856-42a9-a071-8b01d6e95204 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:00.802047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5375d1ac-e856-42a9-a071-8b01d6e95204 None admin] RBAC: Adding query filter params (role.id=e8d679596ffb45939229d1e2a13ce9b0, scope.project.id=f965a93a2f674598890bd8cafededb2c, user.id=dfa2282f957941b98db1d7497da3351b) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:00.802365 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5375d1ac-e856-42a9-a071-8b01d6e95204 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:00.803856 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5375d1ac-e856-42a9-a071-8b01d6e95204 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:00.811737 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 423/870] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:26:00 2022] GET /identity/v3/role_assignments?user.id=dfa2282f957941b98db1d7497da3351b&scope.project.id=f965a93a2f674598890bd8cafededb2c&role.id=e8d679596ffb45939229d1e2a13ce9b0 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:02.370899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8ddf6-09df-48c6-bf79-5af64d223d5a None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:02.371449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8ddf6-09df-48c6-bf79-5af64d223d5a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:02.371449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8ddf6-09df-48c6-bf79-5af64d223d5a None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:02.373065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 448/871] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:02.378520 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46192067-8e33-4f86-9fbc-ae05e15e82de None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:02.378950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46192067-8e33-4f86-9fbc-ae05e15e82de None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:02.379170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46192067-8e33-4f86-9fbc-ae05e15e82de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:02.396663 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-46192067-8e33-4f86-9fbc-ae05e15e82de None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:02.397916 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-46192067-8e33-4f86-9fbc-ae05e15e82de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:02.405752 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:02.476565 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 424/872] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:02.496699 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c8ddf6-09df-48c6-bf79-5af64d223d5a None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:02.610395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c8ddf6-09df-48c6-bf79-5af64d223d5a None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:02.647389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e1d31-71b6-4308-b892-d5862f301122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:02.647977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e1d31-71b6-4308-b892-d5862f301122 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:02.648262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e1d31-71b6-4308-b892-d5862f301122 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:02.648538 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e1d31-71b6-4308-b892-d5862f301122 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:02.649275 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e1d31-71b6-4308-b892-d5862f301122 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:02.650687 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e1d31-71b6-4308-b892-d5862f301122 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:02.674838 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 449/873] 10.222.0.67 () {60 vars in 1141 bytes} [Tue May 31 14:26:02 2022] POST /identity/v3/services => generated 251 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:26:02.934210 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede60194-218d-473f-b9d6-55e88bf4ee0e None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:02.934566 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede60194-218d-473f-b9d6-55e88bf4ee0e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:02.934793 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede60194-218d-473f-b9d6-55e88bf4ee0e None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:02.936397 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 425/874] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:02.942102 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f7f91-8a8f-43db-b06c-90fc27c421d6 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:02.942761 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f7f91-8a8f-43db-b06c-90fc27c421d6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:02.942927 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f7f91-8a8f-43db-b06c-90fc27c421d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:02.944371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706760c-f220-4327-a3ef-a91c8532c81e None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:02.944670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706760c-f220-4327-a3ef-a91c8532c81e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:02.945101 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706760c-f220-4327-a3ef-a91c8532c81e None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:02.946880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 426/875] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 31 14:26:02.956024 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cacabb-20b3-4aa5-8d2c-22a4c626f0a3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:02.956313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cacabb-20b3-4aa5-8d2c-22a4c626f0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:02.956594 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cacabb-20b3-4aa5-8d2c-22a4c626f0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:02.963172 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-d69f7f91-8a8f-43db-b06c-90fc27c421d6 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:02.964381 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d69f7f91-8a8f-43db-b06c-90fc27c421d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:02.976204 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-02cacabb-20b3-4aa5-8d2c-22a4c626f0a3 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:02.977924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-02cacabb-20b3-4aa5-8d2c-22a4c626f0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:03.016485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:03.054688 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 450/876] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:02 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:03.065159 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 427/877] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:02 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 31 14:26:03.077473 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f7f91-8a8f-43db-b06c-90fc27c421d6 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:03.089098 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cacabb-20b3-4aa5-8d2c-22a4c626f0a3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:03.172537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:03.195618 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f7f91-8a8f-43db-b06c-90fc27c421d6 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:03.205211 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cacabb-20b3-4aa5-8d2c-22a4c626f0a3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:03.233389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90de6d-0593-48c0-8193-2c3d5476838e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:03.234165 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90de6d-0593-48c0-8193-2c3d5476838e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:03.234452 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90de6d-0593-48c0-8193-2c3d5476838e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:03.234763 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90de6d-0593-48c0-8193-2c3d5476838e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:03.245908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90de6d-0593-48c0-8193-2c3d5476838e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:26:03.246248 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd8748-3ff3-451a-bb06-10756e3a4231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:03.247170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd8748-3ff3-451a-bb06-10756e3a4231 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:03.247639 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd8748-3ff3-451a-bb06-10756e3a4231 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:03.248154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd8748-3ff3-451a-bb06-10756e3a4231 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:03.249081 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd8748-3ff3-451a-bb06-10756e3a4231 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:03.250630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd8748-3ff3-451a-bb06-10756e3a4231 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:03.254032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-7a90de6d-0593-48c0-8193-2c3d5476838e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 31 14:26:03.254032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:03.254032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:03.254032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:03.254032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:03.254032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:03.254032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:03.254032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:03.254032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:03.254032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:03.254032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:03.254032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:03.254032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:26:03.254032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:26:03.254032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:26:03.254032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:26:03.254993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:26:03.254993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:26:03.254993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:26:03.254993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:26:03.254993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:03.254993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:03.254993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:03.254993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:03.254993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:03.254993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:03.254993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:03.254993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:26:03.254993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:03.254993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:03.254993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:03.254993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:03.255894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:03.255894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:03.255894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:03.255894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:03.255894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:26:03.255894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:26:03.255894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:26:03.255894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:26:03.255894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:26:03.255894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:26:03.255894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 31 14:26:03.255894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:03.255894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 451/878] 10.222.0.67 () {56 vars in 1120 bytes} [Tue May 31 14:26:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:03.257105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b050dc6-4083-4021-93b2-7f8c35e47b0b None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:03.257202 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b050dc6-4083-4021-93b2-7f8c35e47b0b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:03.257470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b050dc6-4083-4021-93b2-7f8c35e47b0b None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:03.258873 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 452/879] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:03.264057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b050dc6-4083-4021-93b2-7f8c35e47b0b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:03.276121 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 428/880] 10.222.0.67 () {60 vars in 1141 bytes} [Tue May 31 14:26:03 2022] POST /identity/v3/services => generated 269 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:26:03.278085 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbcff54-4ffd-4329-85f5-494cd2371acc None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:03.278638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbcff54-4ffd-4329-85f5-494cd2371acc None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:03.279123 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbcff54-4ffd-4329-85f5-494cd2371acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:03.302447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-acbcff54-4ffd-4329-85f5-494cd2371acc None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:03.304037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-acbcff54-4ffd-4329-85f5-494cd2371acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:03.307748 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:03.394878 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b050dc6-4083-4021-93b2-7f8c35e47b0b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:03.406140 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 429/881] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:03.435137 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbcff54-4ffd-4329-85f5-494cd2371acc None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:03.455708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de3314-74e2-4883-8efd-393567a448c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:03.456312 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de3314-74e2-4883-8efd-393567a448c8 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:03.456597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de3314-74e2-4883-8efd-393567a448c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:03.456921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de3314-74e2-4883-8efd-393567a448c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:03.457702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de3314-74e2-4883-8efd-393567a448c8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:03.458011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de3314-74e2-4883-8efd-393567a448c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:03.459547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de3314-74e2-4883-8efd-393567a448c8 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:03.469967 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 453/882] 10.222.0.67 () {56 vars in 1115 bytes} [Tue May 31 14:26:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:03.482683 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de3314-74e2-4883-8efd-393567a448c8 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:03.511429 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:03.563465 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbcff54-4ffd-4329-85f5-494cd2371acc None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:03.600879 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:03.608036 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673889a-ff35-47af-815e-3e2eff4a8fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:03.608870 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673889a-ff35-47af-815e-3e2eff4a8fda None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:03.609159 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673889a-ff35-47af-815e-3e2eff4a8fda None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:03.609442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673889a-ff35-47af-815e-3e2eff4a8fda None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:03.610163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d673889a-ff35-47af-815e-3e2eff4a8fda None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:03.610416 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de3314-74e2-4883-8efd-393567a448c8 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:03.611610 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d673889a-ff35-47af-815e-3e2eff4a8fda None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:03.624605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-d673889a-ff35-47af-815e-3e2eff4a8fda None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 31 14:26:03.624605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:03.624605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:26:03.624605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:26:03.624605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:26:03.624605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:26:03.624605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:26:03.624605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:03.624605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 31 14:26:03.624605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:03.624605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:26:03.624605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:03.624605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:03.624605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:03.624605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:03.624605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:03.624605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:03.625851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:03.625851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:03.625851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:03.625851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:03.625851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:03.625851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:03.625851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:26:03.625851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:26:03.625851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:26:03.625851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:26:03.625851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:03.625851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:03.625851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:03.625851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:03.625851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:03.625851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:03.626904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:03.626904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:03.626904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:03.626904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:03.626904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:03.626904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:03.626904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:03.626904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:03.626904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:03.626904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:03.626904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:26:03.626904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:03.626904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 31 14:26:03.626904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:03.626904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 430/883] 10.222.0.67 () {56 vars in 1126 bytes} [Tue May 31 14:26:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:26:03.632143 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673889a-ff35-47af-815e-3e2eff4a8fda None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:03.655977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a1be4-9b01-4a34-9f13-01e1f62432c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:03.657368 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a1be4-9b01-4a34-9f13-01e1f62432c6 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:03.658065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a1be4-9b01-4a34-9f13-01e1f62432c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:03.658972 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a1be4-9b01-4a34-9f13-01e1f62432c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:03.673564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a1be4-9b01-4a34-9f13-01e1f62432c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:03.675748 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a1be4-9b01-4a34-9f13-01e1f62432c6 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:03.701457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-420a1be4-9b01-4a34-9f13-01e1f62432c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:03.701457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:03.701457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:03.701457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:03.701457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:03.701457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:03.701457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:03.701457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:03.701457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:03.701457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:03.701457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:03.701457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:03.701457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:26:03.701457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:26:03.701457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:26:03.702597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:26:03.702597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:03.702597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:03.702597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:03.702597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:03.702597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:03.702597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:03.702597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:03.702597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:26:03.702597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:03.702597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:03.702597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:03.702597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:03.702597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:03.702597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:03.702597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:03.702597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:03.703554 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:26:03.703554 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:26:03.703554 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:03.703554 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:03.704885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 454/884] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:26:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:26:03.710758 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a1be4-9b01-4a34-9f13-01e1f62432c6 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:03.753929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673889a-ff35-47af-815e-3e2eff4a8fda None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:03.782227 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:03.793476 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c897b32-9bfc-4ce3-b967-0f2727b7bd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:03.794488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c897b32-9bfc-4ce3-b967-0f2727b7bd11 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:03.795029 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c897b32-9bfc-4ce3-b967-0f2727b7bd11 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:03.795514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c897b32-9bfc-4ce3-b967-0f2727b7bd11 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:03.796430 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c897b32-9bfc-4ce3-b967-0f2727b7bd11 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:03.796956 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c897b32-9bfc-4ce3-b967-0f2727b7bd11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:03.798546 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c897b32-9bfc-4ce3-b967-0f2727b7bd11 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:03.810080 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 431/885] 10.222.0.67 () {56 vars in 1122 bytes} [Tue May 31 14:26:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:03.811317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c897b32-9bfc-4ce3-b967-0f2727b7bd11 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:03.828251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a1be4-9b01-4a34-9f13-01e1f62432c6 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:03.874758 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032f4c3-7d47-4307-b25e-1d467ccf7974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:03.875614 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032f4c3-7d47-4307-b25e-1d467ccf7974 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:03.875905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032f4c3-7d47-4307-b25e-1d467ccf7974 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:03.876185 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032f4c3-7d47-4307-b25e-1d467ccf7974 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:03.877221 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032f4c3-7d47-4307-b25e-1d467ccf7974 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:03.877556 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032f4c3-7d47-4307-b25e-1d467ccf7974 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:03.879237 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032f4c3-7d47-4307-b25e-1d467ccf7974 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:03.895872 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 455/886] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:03.897337 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032f4c3-7d47-4307-b25e-1d467ccf7974 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:03.907399 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:03.929203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c897b32-9bfc-4ce3-b967-0f2727b7bd11 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:03.967385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9dc93d-5adc-4fb3-bb01-8d7a26b94a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:03.968397 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9dc93d-5adc-4fb3-bb01-8d7a26b94a8c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:03.968923 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9dc93d-5adc-4fb3-bb01-8d7a26b94a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:03.969413 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9dc93d-5adc-4fb3-bb01-8d7a26b94a8c None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:03.970331 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9dc93d-5adc-4fb3-bb01-8d7a26b94a8c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:03.970869 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9dc93d-5adc-4fb3-bb01-8d7a26b94a8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:03.972489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9dc93d-5adc-4fb3-bb01-8d7a26b94a8c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:03.982945 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 432/887] 10.222.0.67 () {56 vars in 1122 bytes} [Tue May 31 14:26:03 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:04.008574 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032f4c3-7d47-4307-b25e-1d467ccf7974 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:04.048252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27699d-f1e7-4946-9b2c-fcf9e26e4e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:04.049105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27699d-f1e7-4946-9b2c-fcf9e26e4e2a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:04.049388 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27699d-f1e7-4946-9b2c-fcf9e26e4e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:04.049665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27699d-f1e7-4946-9b2c-fcf9e26e4e2a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:04.058968 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27699d-f1e7-4946-9b2c-fcf9e26e4e2a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:04.060470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27699d-f1e7-4946-9b2c-fcf9e26e4e2a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:04.077444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-ae27699d-f1e7-4946-9b2c-fcf9e26e4e2a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 31 14:26:04.077444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:04.077444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:04.077444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:04.077444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:04.077444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:04.077444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:04.077444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:04.077444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:04.077444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:04.077444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:04.077444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:04.077444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:26:04.077444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:26:04.077444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:26:04.077444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:26:04.078491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:04.078491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:04.078491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:26:04.078491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:26:04.078491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:26:04.078491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:26:04.078491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:04.078491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:04.078491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:04.078491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:04.078491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:04.078491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:26:04.078491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:04.078491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:04.078491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:04.078491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:04.078491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:04.078491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:04.079500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:04.079500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:04.079500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:26:04.079500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:26:04.079500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:26:04.079500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:26:04.079500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:26:04.079500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:26:04.079500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 31 14:26:04.079500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:04.082759 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 456/888] 10.222.0.67 () {56 vars in 1117 bytes} [Tue May 31 14:26:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:04.085098 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9dc93d-5adc-4fb3-bb01-8d7a26b94a8c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:04.146602 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:04.195761 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9dc93d-5adc-4fb3-bb01-8d7a26b94a8c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:04.231787 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fcbbbd-4325-4709-a997-1e1bebf30c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:04.232810 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcbbbd-4325-4709-a997-1e1bebf30c2d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:04.233305 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcbbbd-4325-4709-a997-1e1bebf30c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:04.233787 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcbbbd-4325-4709-a997-1e1bebf30c2d None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:04.234942 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fcbbbd-4325-4709-a997-1e1bebf30c2d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:04.235485 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fcbbbd-4325-4709-a997-1e1bebf30c2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:04.237132 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fcbbbd-4325-4709-a997-1e1bebf30c2d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:04.255399 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 433/889] 10.222.0.67 () {56 vars in 1149 bytes} [Tue May 31 14:26:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:04.262668 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27699d-f1e7-4946-9b2c-fcf9e26e4e2a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:04.331749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:04.368242 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27699d-f1e7-4946-9b2c-fcf9e26e4e2a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:04.403374 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75951fef-91e8-4989-8cd6-aab37fd71445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:04.404023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75951fef-91e8-4989-8cd6-aab37fd71445 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:04.404306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75951fef-91e8-4989-8cd6-aab37fd71445 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:04.404586 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75951fef-91e8-4989-8cd6-aab37fd71445 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:04.474110 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75951fef-91e8-4989-8cd6-aab37fd71445 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:04.475604 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75951fef-91e8-4989-8cd6-aab37fd71445 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:04.577459 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:04.646791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 457/890] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:26:04 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 31 14:26:04.651743 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fcbbbd-4325-4709-a997-1e1bebf30c2d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:04.767130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fcbbbd-4325-4709-a997-1e1bebf30c2d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:04.780919 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:04.802980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05e7aa-f15d-4a50-9df6-a0921283a60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:04.803791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05e7aa-f15d-4a50-9df6-a0921283a60a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:04.804072 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05e7aa-f15d-4a50-9df6-a0921283a60a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:04.804343 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05e7aa-f15d-4a50-9df6-a0921283a60a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:04.815376 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05e7aa-f15d-4a50-9df6-a0921283a60a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:04.816809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05e7aa-f15d-4a50-9df6-a0921283a60a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:04.834768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-4d05e7aa-f15d-4a50-9df6-a0921283a60a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:04.834768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:04.834768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:04.834768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:04.834768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:04.834768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:04.834768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:04.834768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:04.834768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:04.834768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:04.834768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:04.834768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:04.834768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:26:04.834768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:26:04.834768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:26:04.835669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:26:04.835669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:04.835669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:04.835669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:04.835669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:04.835669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:04.835669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:04.835669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:04.835669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:04.835669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:04.835669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:04.835669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:04.835669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:04.835669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:04.835669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:04.835669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:04.835669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:04.836780 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:26:04.836780 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:26:04.836780 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:04.836780 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:04.836780 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 434/891] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:26:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:04.843091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75951fef-91e8-4989-8cd6-aab37fd71445 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:04.945310 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75951fef-91e8-4989-8cd6-aab37fd71445 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:04.980634 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28499bd7-0895-46e3-9931-be8614e3ea11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:04.981628 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28499bd7-0895-46e3-9931-be8614e3ea11 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:04.982135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28499bd7-0895-46e3-9931-be8614e3ea11 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:04.982618 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28499bd7-0895-46e3-9931-be8614e3ea11 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:04.983564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28499bd7-0895-46e3-9931-be8614e3ea11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:04.984095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28499bd7-0895-46e3-9931-be8614e3ea11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:04.985669 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28499bd7-0895-46e3-9931-be8614e3ea11 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:05.002542 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 458/892] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:05.004167 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05e7aa-f15d-4a50-9df6-a0921283a60a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:05.051370 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a718061-40c2-4e30-972e-9ad8b3343962 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:05.051712 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a718061-40c2-4e30-972e-9ad8b3343962 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:05.051974 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a718061-40c2-4e30-972e-9ad8b3343962 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:05.053785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 459/893] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 31 14:26:05.059272 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92412812-bb4f-4165-a97f-e7371dc9b420 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:05.059770 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92412812-bb4f-4165-a97f-e7371dc9b420 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:05.059895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92412812-bb4f-4165-a97f-e7371dc9b420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:05.077010 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-92412812-bb4f-4165-a97f-e7371dc9b420 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:05.078203 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-92412812-bb4f-4165-a97f-e7371dc9b420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:05.107078 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:05.126769 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05e7aa-f15d-4a50-9df6-a0921283a60a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:05.158967 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 460/894] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 31 14:26:05.171183 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c53e7-6a34-432c-b977-be5821eddf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:05.172785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c53e7-6a34-432c-b977-be5821eddf94 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:05.173281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c53e7-6a34-432c-b977-be5821eddf94 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:05.173667 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c53e7-6a34-432c-b977-be5821eddf94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:05.177388 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92412812-bb4f-4165-a97f-e7371dc9b420 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:05.183408 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c53e7-6a34-432c-b977-be5821eddf94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:05.184870 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c53e7-6a34-432c-b977-be5821eddf94 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:05.202762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-304c53e7-6a34-432c-b977-be5821eddf94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:26:05.202762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:05.202762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:05.202762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:05.202762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:05.202762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:05.202762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:05.202762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:05.202762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:05.202762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:05.202762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:05.202762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:05.202762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:26:05.202762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:26:05.202762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:26:05.203687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:26:05.203687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:05.203687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:05.203687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:05.203687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:05.203687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:05.203687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:05.203687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:05.203687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:05.203687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:05.203687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:05.203687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:05.203687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:05.203687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:05.203687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:05.203687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:05.203687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:05.204575 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:26:05.204575 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:26:05.204575 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:26:05.204575 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:26:05.204575 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:26:05.204575 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:26:05.204575 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:26:05.204575 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:05.207035 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 435/895] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:26:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:05.213755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c53e7-6a34-432c-b977-be5821eddf94 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:05.247942 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:05.292836 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92412812-bb4f-4165-a97f-e7371dc9b420 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:05.328440 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832b8b4-1fd0-4163-aa57-0fe506ccc1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:05.329656 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832b8b4-1fd0-4163-aa57-0fe506ccc1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:05.330213 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832b8b4-1fd0-4163-aa57-0fe506ccc1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:05.330707 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832b8b4-1fd0-4163-aa57-0fe506ccc1f4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:05.331610 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832b8b4-1fd0-4163-aa57-0fe506ccc1f4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:05.333198 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832b8b4-1fd0-4163-aa57-0fe506ccc1f4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:05.346438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-0832b8b4-1fd0-4163-aa57-0fe506ccc1f4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 31 14:26:05.346438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:05.346438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:26:05.346438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:26:05.346438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:26:05.346438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:26:05.346438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:26:05.346438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:05.346438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 31 14:26:05.346438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:05.346438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:26:05.346438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:05.346438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:05.346438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:05.346438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:05.346438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:05.346438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:05.347659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:05.347659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:05.347659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:05.347659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:05.347659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:05.347659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:05.347659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:26:05.347659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:26:05.347659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:26:05.347659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:26:05.347659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:05.347659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:05.347659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:05.347659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:05.347659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:05.347659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:05.348500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:05.348500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:26:05.348500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:05.348500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:05.348500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:05.348500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:05.348500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:05.348500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:05.348500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:05.348500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:05.348500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:26:05.348500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:05.348500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 31 14:26:05.348500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:05.349239 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c53e7-6a34-432c-b977-be5821eddf94 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:05.356676 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 461/896] 10.222.0.67 () {56 vars in 1114 bytes} [Tue May 31 14:26:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:26:05.357844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832b8b4-1fd0-4163-aa57-0fe506ccc1f4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:05.385284 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:05.388205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f27aa-001a-47cc-8ccd-3b54f4777ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:05.388932 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f27aa-001a-47cc-8ccd-3b54f4777ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:05.389296 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f27aa-001a-47cc-8ccd-3b54f4777ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:05.389504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f27aa-001a-47cc-8ccd-3b54f4777ab7 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:05.390305 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f27aa-001a-47cc-8ccd-3b54f4777ab7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:05.390621 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f27aa-001a-47cc-8ccd-3b54f4777ab7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:05.392075 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f27aa-001a-47cc-8ccd-3b54f4777ab7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:05.410226 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 436/897] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:26:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:05.412002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f27aa-001a-47cc-8ccd-3b54f4777ab7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:05.487465 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832b8b4-1fd0-4163-aa57-0fe506ccc1f4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:05.502483 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:05.521112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f27aa-001a-47cc-8ccd-3b54f4777ab7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:05.526876 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48caf45d-73fa-4a10-83ca-1e1045709058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:05.527809 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48caf45d-73fa-4a10-83ca-1e1045709058 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:05.528287 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48caf45d-73fa-4a10-83ca-1e1045709058 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:05.528754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48caf45d-73fa-4a10-83ca-1e1045709058 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:05.529757 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48caf45d-73fa-4a10-83ca-1e1045709058 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:05.530259 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48caf45d-73fa-4a10-83ca-1e1045709058 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:05.532001 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48caf45d-73fa-4a10-83ca-1e1045709058 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:05.542210 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 462/898] 10.222.0.67 () {56 vars in 1114 bytes} [Tue May 31 14:26:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:05.543433 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48caf45d-73fa-4a10-83ca-1e1045709058 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:05.557537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59c43f-fe2d-47de-b6e9-3f2f3e9351fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:05.558183 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59c43f-fe2d-47de-b6e9-3f2f3e9351fd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:05.558476 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59c43f-fe2d-47de-b6e9-3f2f3e9351fd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:05.558851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59c43f-fe2d-47de-b6e9-3f2f3e9351fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:05.559689 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e59c43f-fe2d-47de-b6e9-3f2f3e9351fd None admin] RBAC: Adding query filter params (role.id=8de91fffa27842109896bf18045db796, scope.project.id=f965a93a2f674598890bd8cafededb2c, user.id=0b7d100f2407432381d119c25814fff3) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:05.560004 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e59c43f-fe2d-47de-b6e9-3f2f3e9351fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:05.561499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e59c43f-fe2d-47de-b6e9-3f2f3e9351fd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:05.574837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 437/899] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:26:05 2022] GET /identity/v3/role_assignments?user.id=0b7d100f2407432381d119c25814fff3&scope.project.id=f965a93a2f674598890bd8cafededb2c&role.id=8de91fffa27842109896bf18045db796 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:05.642239 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:05.661611 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48caf45d-73fa-4a10-83ca-1e1045709058 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:05.699443 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c76e1-bce5-4165-ad39-6a6eebc559da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:05.700502 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c76e1-bce5-4165-ad39-6a6eebc559da None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:05.701249 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c76e1-bce5-4165-ad39-6a6eebc559da None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:05.701721 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c76e1-bce5-4165-ad39-6a6eebc559da None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:05.702685 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c76e1-bce5-4165-ad39-6a6eebc559da None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:05.703194 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c76e1-bce5-4165-ad39-6a6eebc559da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:05.704796 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c76e1-bce5-4165-ad39-6a6eebc559da None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:05.711155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 463/900] 10.222.0.67 () {56 vars in 1114 bytes} [Tue May 31 14:26:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:05.717874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59c43f-fe2d-47de-b6e9-3f2f3e9351fd None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:05.767778 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a15e2f-656b-4671-a165-d8f5e70e6b43 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:05.768411 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a15e2f-656b-4671-a165-d8f5e70e6b43 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:05.768923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a15e2f-656b-4671-a165-d8f5e70e6b43 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:05.770966 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 464/901] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:05.776496 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96dc2d-0d9c-433a-a677-697a526e375b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:05.776780 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96dc2d-0d9c-433a-a677-697a526e375b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:05.777052 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96dc2d-0d9c-433a-a677-697a526e375b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:05.795713 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6f96dc2d-0d9c-433a-a677-697a526e375b None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:05.796979 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6f96dc2d-0d9c-433a-a677-697a526e375b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:05.841824 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59c43f-fe2d-47de-b6e9-3f2f3e9351fd None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:05.877736 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 465/902] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:05.879625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b68d5-e383-4b0c-b686-3a9ea8af8021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:05.880303 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b68d5-e383-4b0c-b686-3a9ea8af8021 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:05.880595 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b68d5-e383-4b0c-b686-3a9ea8af8021 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:05.880878 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b68d5-e383-4b0c-b686-3a9ea8af8021 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:05.881725 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b68d5-e383-4b0c-b686-3a9ea8af8021 None admin] RBAC: Adding query filter params (interface=public, service_id=4621cdae749f44b083bd80cda84c4d0e, region_id=RegionOne) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:05.882055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b68d5-e383-4b0c-b686-3a9ea8af8021 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:05.883507 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b68d5-e383-4b0c-b686-3a9ea8af8021 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:05.889746 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 438/903] 10.222.0.67 () {56 vars in 1251 bytes} [Tue May 31 14:26:05 2022] GET /identity/v3/endpoints?service_id=4621cdae749f44b083bd80cda84c4d0e&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:05.894837 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96dc2d-0d9c-433a-a677-697a526e375b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:05.903098 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b68d5-e383-4b0c-b686-3a9ea8af8021 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:05.923025 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:05.987275 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:06.013562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96dc2d-0d9c-433a-a677-697a526e375b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:06.019832 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b68d5-e383-4b0c-b686-3a9ea8af8021 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:06.051044 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c61d79-df29-4358-ac78-01bf873de409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:06.052066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c61d79-df29-4358-ac78-01bf873de409 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:06.052555 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c61d79-df29-4358-ac78-01bf873de409 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:06.053051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c61d79-df29-4358-ac78-01bf873de409 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:06.053968 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c61d79-df29-4358-ac78-01bf873de409 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:06.055573 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c61d79-df29-4358-ac78-01bf873de409 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:06.061924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b4359-2c86-40e7-bef0-8d231818efcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:06.062793 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b4359-2c86-40e7-bef0-8d231818efcd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:06.063078 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b4359-2c86-40e7-bef0-8d231818efcd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:06.063350 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b4359-2c86-40e7-bef0-8d231818efcd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:06.064063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b4359-2c86-40e7-bef0-8d231818efcd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:06.064267 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 466/904] 10.222.0.67 () {56 vars in 1087 bytes} [Tue May 31 14:26:05 2022] GET /identity/v3/services => generated 1268 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:26:06.065425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b4359-2c86-40e7-bef0-8d231818efcd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:06.086773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-2c9b4359-2c86-40e7-bef0-8d231818efcd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 31 14:26:06.086773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:06.086773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:26:06.086773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:26:06.086773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:26:06.086773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:26:06.086773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:26:06.086773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:06.086773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 31 14:26:06.086773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:06.086773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:26:06.086773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:06.086773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:06.086773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:06.086773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:06.086773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:06.087791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:06.087791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:06.087791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:06.087791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:06.087791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:06.087791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:06.087791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:06.087791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:26:06.087791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:26:06.087791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:26:06.087791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:26:06.087791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:06.087791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:06.087791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:06.087791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:06.087791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:06.088686 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:06.088686 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:06.088686 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:06.088686 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:06.088686 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:06.088686 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:06.088686 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:06.088686 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:06.088686 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:06.088686 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:06.088686 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:06.088686 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:26:06.088686 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:06.088686 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 31 14:26:06.088686 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:06.091899 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 439/905] 10.222.0.67 () {56 vars in 1132 bytes} [Tue May 31 14:26:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:26:06.098329 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c61d79-df29-4358-ac78-01bf873de409 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:06.217891 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c61d79-df29-4358-ac78-01bf873de409 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:06.257544 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8c6ac-511b-41c9-8397-66ceeefefc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:06.258232 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8c6ac-511b-41c9-8397-66ceeefefc1d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:06.258521 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8c6ac-511b-41c9-8397-66ceeefefc1d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:06.258834 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8c6ac-511b-41c9-8397-66ceeefefc1d None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:06.259565 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b8c6ac-511b-41c9-8397-66ceeefefc1d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:06.259918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b8c6ac-511b-41c9-8397-66ceeefefc1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:06.261330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b8c6ac-511b-41c9-8397-66ceeefefc1d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:06.267087 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 467/906] 10.222.0.67 () {56 vars in 1126 bytes} [Tue May 31 14:26:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:06.272741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b4359-2c86-40e7-bef0-8d231818efcd None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:06.316011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605358f-9533-4526-ad6d-0fd9ab5126e9 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:06.316011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605358f-9533-4526-ad6d-0fd9ab5126e9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:06.316011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605358f-9533-4526-ad6d-0fd9ab5126e9 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:06.318789 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 468/907] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:06.327268 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3afb4c1-0ab9-4c56-b47d-99fa6be855ff None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:06.327558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3afb4c1-0ab9-4c56-b47d-99fa6be855ff None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:06.327826 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3afb4c1-0ab9-4c56-b47d-99fa6be855ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:06.344203 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:06.347278 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-c3afb4c1-0ab9-4c56-b47d-99fa6be855ff None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:06.348894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c3afb4c1-0ab9-4c56-b47d-99fa6be855ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:06.364173 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:06.398174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b4359-2c86-40e7-bef0-8d231818efcd None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:06.436836 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62527e89-70ec-4b86-8a9b-1b96092eb03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:06.437540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62527e89-70ec-4b86-8a9b-1b96092eb03d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:06.437829 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62527e89-70ec-4b86-8a9b-1b96092eb03d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:06.438111 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62527e89-70ec-4b86-8a9b-1b96092eb03d None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:06.438936 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62527e89-70ec-4b86-8a9b-1b96092eb03d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:06.439257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62527e89-70ec-4b86-8a9b-1b96092eb03d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:06.440631 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62527e89-70ec-4b86-8a9b-1b96092eb03d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:06.441680 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 469/908] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:06.446132 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 440/909] 10.222.0.67 () {56 vars in 1126 bytes} [Tue May 31 14:26:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:06.452082 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3afb4c1-0ab9-4c56-b47d-99fa6be855ff None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:06.461466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62527e89-70ec-4b86-8a9b-1b96092eb03d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:06.572541 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62527e89-70ec-4b86-8a9b-1b96092eb03d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:06.581476 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3afb4c1-0ab9-4c56-b47d-99fa6be855ff None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:06.610295 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16b09d-8c6b-41a4-93b4-edfd993e10fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:06.611447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16b09d-8c6b-41a4-93b4-edfd993e10fb None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:06.611935 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16b09d-8c6b-41a4-93b4-edfd993e10fb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:06.612433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16b09d-8c6b-41a4-93b4-edfd993e10fb None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:06.613379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16b09d-8c6b-41a4-93b4-edfd993e10fb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:06.615309 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16b09d-8c6b-41a4-93b4-edfd993e10fb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:06.626540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:06.626817 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a872f09-6bf8-405e-a021-6ef58ccd9fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:06.628318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a872f09-6bf8-405e-a021-6ef58ccd9fcc None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:06.628611 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a872f09-6bf8-405e-a021-6ef58ccd9fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:06.628993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a872f09-6bf8-405e-a021-6ef58ccd9fcc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:06.630143 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a872f09-6bf8-405e-a021-6ef58ccd9fcc None admin] RBAC: Adding query filter params (interface=public, service_id=a80e7f7de56f427eb755dd19757e0db5, region_id=RegionOne) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:06.630615 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a872f09-6bf8-405e-a021-6ef58ccd9fcc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:06.632814 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a872f09-6bf8-405e-a021-6ef58ccd9fcc None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:06.643098 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:06.643659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 470/911] 10.222.0.67 () {56 vars in 1251 bytes} [Tue May 31 14:26:06 2022] GET /identity/v3/endpoints?service_id=a80e7f7de56f427eb755dd19757e0db5&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:06.644499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 441/911] 10.222.0.67 () {60 vars in 1141 bytes} [Tue May 31 14:26:06 2022] POST /identity/v3/services => generated 241 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:26:06.644984 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a872f09-6bf8-405e-a021-6ef58ccd9fcc None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:06.763882 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a872f09-6bf8-405e-a021-6ef58ccd9fcc None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:06.810191 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753c9f3-fa62-4328-8943-23b3a5852993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:06.810895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753c9f3-fa62-4328-8943-23b3a5852993 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:06.811176 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753c9f3-fa62-4328-8943-23b3a5852993 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:06.811447 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753c9f3-fa62-4328-8943-23b3a5852993 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:06.812138 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753c9f3-fa62-4328-8943-23b3a5852993 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:06.813584 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753c9f3-fa62-4328-8943-23b3a5852993 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:06.820672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 471/912] 10.222.0.67 () {56 vars in 1087 bytes} [Tue May 31 14:26:06 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:26:07.943777 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905931d4-aa64-4bfb-b1b9-32dfeb083cfc None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:07.944302 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905931d4-aa64-4bfb-b1b9-32dfeb083cfc None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:07.944695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905931d4-aa64-4bfb-b1b9-32dfeb083cfc None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:07.947789 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 442/913] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:07.956067 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54f61e-c2de-492e-a39a-d0978d784a4f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:07.956625 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54f61e-c2de-492e-a39a-d0978d784a4f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:07.956898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54f61e-c2de-492e-a39a-d0978d784a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:07.979583 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-0a54f61e-c2de-492e-a39a-d0978d784a4f None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:07.980817 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0a54f61e-c2de-492e-a39a-d0978d784a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:08.041170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:08.058761 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 472/914] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:08.077053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905931d4-aa64-4bfb-b1b9-32dfeb083cfc None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:08.195173 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905931d4-aa64-4bfb-b1b9-32dfeb083cfc None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:08.229961 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73478d5c-c6f4-4065-9e24-041c04e97d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:08.230852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73478d5c-c6f4-4065-9e24-041c04e97d7b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:08.231143 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73478d5c-c6f4-4065-9e24-041c04e97d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:08.231418 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73478d5c-c6f4-4065-9e24-041c04e97d7b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:08.243060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73478d5c-c6f4-4065-9e24-041c04e97d7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:26:08.251857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-73478d5c-c6f4-4065-9e24-041c04e97d7b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 31 14:26:08.251857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:08.251857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:08.251857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:08.251857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:08.251857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:08.251857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:08.251857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:08.251857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:08.251857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:08.251857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:08.251857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:08.251857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:26:08.251857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:26:08.251857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:26:08.251857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:26:08.252792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:26:08.252792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:26:08.252792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:26:08.252792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:26:08.252792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:08.252792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:08.252792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:08.252792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:08.252792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:08.252792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:08.252792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:08.252792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:08.252792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:08.252792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:08.252792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:08.252792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:08.253924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:08.253924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:08.253924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:08.253924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:08.253924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:26:08.253924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:26:08.253924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:26:08.253924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:26:08.253924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:26:08.253924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:26:08.253924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 31 14:26:08.253924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:08.253924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 443/915] 10.222.0.67 () {56 vars in 1120 bytes} [Tue May 31 14:26:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:08.261319 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a54f61e-c2de-492e-a39a-d0978d784a4f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:08.368629 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a54f61e-c2de-492e-a39a-d0978d784a4f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:08.403623 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696ff97-6151-47bb-93a4-8bf8127c40ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:08.404293 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696ff97-6151-47bb-93a4-8bf8127c40ab None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:08.404576 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696ff97-6151-47bb-93a4-8bf8127c40ab None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:08.404848 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696ff97-6151-47bb-93a4-8bf8127c40ab None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:08.405598 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696ff97-6151-47bb-93a4-8bf8127c40ab None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:08.405905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696ff97-6151-47bb-93a4-8bf8127c40ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:08.407311 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696ff97-6151-47bb-93a4-8bf8127c40ab None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:08.416037 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 473/916] 10.222.0.67 () {56 vars in 1115 bytes} [Tue May 31 14:26:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:08.418955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ab7f9-cc52-42c5-9ea0-b75e97547163 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:08.419250 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ab7f9-cc52-42c5-9ea0-b75e97547163 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:08.419533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ab7f9-cc52-42c5-9ea0-b75e97547163 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:08.421198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 444/917] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:08.421601 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696ff97-6151-47bb-93a4-8bf8127c40ab None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:08.426080 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb131ee-456d-4f29-8789-ffd554b0a4b0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:08.426374 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb131ee-456d-4f29-8789-ffd554b0a4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:08.426675 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb131ee-456d-4f29-8789-ffd554b0a4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:08.444045 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7fb131ee-456d-4f29-8789-ffd554b0a4b0 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:08.445301 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7fb131ee-456d-4f29-8789-ffd554b0a4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:08.495204 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:08.510162 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:08.533711 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 445/918] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 31 14:26:08.552039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696ff97-6151-47bb-93a4-8bf8127c40ab None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:08.558290 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb131ee-456d-4f29-8789-ffd554b0a4b0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:08.589766 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ade5ca-f3c4-4b97-b9f9-a6a2592c3031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:08.590809 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ade5ca-f3c4-4b97-b9f9-a6a2592c3031 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:08.591200 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ade5ca-f3c4-4b97-b9f9-a6a2592c3031 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:08.591479 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ade5ca-f3c4-4b97-b9f9-a6a2592c3031 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:08.603307 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ade5ca-f3c4-4b97-b9f9-a6a2592c3031 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:08.605373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ade5ca-f3c4-4b97-b9f9-a6a2592c3031 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:08.625014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-b6ade5ca-f3c4-4b97-b9f9-a6a2592c3031 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:08.625014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:08.625014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:08.625014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:08.625014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:08.625014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:08.625014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:08.625014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:08.625014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:08.625014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:08.625014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:08.625014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:08.625014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:26:08.625014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:26:08.625014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:26:08.626104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:26:08.626104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:08.626104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:08.626104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:08.626104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:08.626104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:08.626104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:08.626104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:08.626104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:26:08.626104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:08.626104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:08.626104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:08.626104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:08.626104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:08.626104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:08.626104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:08.626104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:08.627234 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:26:08.627234 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:26:08.627234 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:08.627234 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:08.627234 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 474/919] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:26:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:26:08.635044 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ade5ca-f3c4-4b97-b9f9-a6a2592c3031 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:08.690923 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb131ee-456d-4f29-8789-ffd554b0a4b0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:08.736196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a6aec-4f60-4661-af0f-3b49901cf54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:08.737227 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a6aec-4f60-4661-af0f-3b49901cf54e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:08.737644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a6aec-4f60-4661-af0f-3b49901cf54e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:08.738077 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a6aec-4f60-4661-af0f-3b49901cf54e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:08.738900 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a6aec-4f60-4661-af0f-3b49901cf54e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:08.740399 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a6aec-4f60-4661-af0f-3b49901cf54e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:08.753454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-298a6aec-4f60-4661-af0f-3b49901cf54e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 31 14:26:08.753454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:08.753454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:26:08.753454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:26:08.753454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:26:08.753454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:26:08.753454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:26:08.753454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:08.753454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 31 14:26:08.753454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:08.753454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:26:08.753454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:08.753454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:08.753454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:08.753454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:08.753454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:08.753454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:08.754638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:08.754638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:08.754638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:08.754638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:08.754638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:08.754638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:08.754638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:26:08.754638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:26:08.754638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:26:08.754638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:26:08.754638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:08.754638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:08.754638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:08.754638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:08.754638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:08.754638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:08.755701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:08.755701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:08.755701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:08.755701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:08.755701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:08.755701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:08.755701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:08.755701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:08.755701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:08.755701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:08.755701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:26:08.755701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:08.755701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 31 14:26:08.755701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:08.758366 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 446/920] 10.222.0.67 () {56 vars in 1114 bytes} [Tue May 31 14:26:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:08.764888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a6aec-4f60-4661-af0f-3b49901cf54e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:08.766690 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ade5ca-f3c4-4b97-b9f9-a6a2592c3031 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:08.770548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:08.807452 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c09c78-f2c1-4d6b-89a9-7e6c91f50baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:08.808454 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c09c78-f2c1-4d6b-89a9-7e6c91f50baa None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:08.808879 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c09c78-f2c1-4d6b-89a9-7e6c91f50baa None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:08.809320 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c09c78-f2c1-4d6b-89a9-7e6c91f50baa None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:08.810192 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c09c78-f2c1-4d6b-89a9-7e6c91f50baa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:08.810712 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c09c78-f2c1-4d6b-89a9-7e6c91f50baa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:08.812370 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c09c78-f2c1-4d6b-89a9-7e6c91f50baa None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:08.831226 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 475/921] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:08.832302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c09c78-f2c1-4d6b-89a9-7e6c91f50baa None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:08.835976 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:08.909051 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a6aec-4f60-4661-af0f-3b49901cf54e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:08.945854 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c09c78-f2c1-4d6b-89a9-7e6c91f50baa None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:08.956098 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc57ac-1616-461d-801b-3345b72839f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:08.957666 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc57ac-1616-461d-801b-3345b72839f0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:08.958075 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc57ac-1616-461d-801b-3345b72839f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:08.958471 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc57ac-1616-461d-801b-3345b72839f0 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:08.959760 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc57ac-1616-461d-801b-3345b72839f0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:08.960457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc57ac-1616-461d-801b-3345b72839f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:08.962179 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc57ac-1616-461d-801b-3345b72839f0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:08.970819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 447/922] 10.222.0.67 () {56 vars in 1114 bytes} [Tue May 31 14:26:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:08.975095 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc57ac-1616-461d-801b-3345b72839f0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:08.994897 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58629f79-f4a3-4f2b-b3c0-d9d13525fb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:08.996897 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58629f79-f4a3-4f2b-b3c0-d9d13525fb36 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:08.997336 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58629f79-f4a3-4f2b-b3c0-d9d13525fb36 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:08.997774 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58629f79-f4a3-4f2b-b3c0-d9d13525fb36 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:09.007281 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58629f79-f4a3-4f2b-b3c0-d9d13525fb36 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:09.008751 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58629f79-f4a3-4f2b-b3c0-d9d13525fb36 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:09.023769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-58629f79-f4a3-4f2b-b3c0-d9d13525fb36 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 31 14:26:09.023769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:09.023769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:09.023769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:09.023769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:09.023769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:09.023769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:09.023769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:09.023769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:09.023769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:09.023769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:09.023769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:09.023769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:26:09.023769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:26:09.023769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:26:09.023769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:26:09.024747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:09.024747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:09.024747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:26:09.024747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:26:09.024747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:26:09.024747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:26:09.024747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:09.024747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:09.024747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:09.024747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:09.024747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:09.024747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:26:09.024747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:09.024747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:09.024747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:09.024747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:09.024747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:09.024747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:09.026243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:09.026243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:09.026243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:26:09.026243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:26:09.026243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:26:09.026243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:26:09.026243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:26:09.026243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:26:09.026243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 31 14:26:09.026243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:09.026243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 476/923] 10.222.0.67 () {56 vars in 1117 bytes} [Tue May 31 14:26:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:09.027540 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d3116-be7d-40c5-b982-5f313f0861bf None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:09.027540 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d3116-be7d-40c5-b982-5f313f0861bf None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:09.027540 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d3116-be7d-40c5-b982-5f313f0861bf None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:09.028728 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 477/924] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:09.030845 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d3116-be7d-40c5-b982-5f313f0861bf None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:09.059344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:09.087515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc57ac-1616-461d-801b-3345b72839f0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:09.106428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:09.132828 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8c189-553b-4a37-b681-fdcb326f97e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:09.133640 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8c189-553b-4a37-b681-fdcb326f97e1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:09.133937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8c189-553b-4a37-b681-fdcb326f97e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:09.134215 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8c189-553b-4a37-b681-fdcb326f97e1 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:09.134988 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8c189-553b-4a37-b681-fdcb326f97e1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:09.135304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8c189-553b-4a37-b681-fdcb326f97e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:09.136692 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8c189-553b-4a37-b681-fdcb326f97e1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:09.141860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 448/925] 10.222.0.67 () {56 vars in 1114 bytes} [Tue May 31 14:26:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:09.142942 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c2de3-308e-435b-86df-e8b6cb45741c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:09.143226 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c2de3-308e-435b-86df-e8b6cb45741c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:09.143529 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c2de3-308e-435b-86df-e8b6cb45741c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:09.154097 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d3116-be7d-40c5-b982-5f313f0861bf None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:09.160221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b15c2de3-308e-435b-86df-e8b6cb45741c None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:09.161393 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b15c2de3-308e-435b-86df-e8b6cb45741c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:09.192309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc73302f-9642-4aff-b176-7ba36201d08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:09.192978 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73302f-9642-4aff-b176-7ba36201d08e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:09.193264 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73302f-9642-4aff-b176-7ba36201d08e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:09.193544 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73302f-9642-4aff-b176-7ba36201d08e None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:09.194379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc73302f-9642-4aff-b176-7ba36201d08e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:09.194760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc73302f-9642-4aff-b176-7ba36201d08e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:09.196190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc73302f-9642-4aff-b176-7ba36201d08e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:09.219841 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 478/926] 10.222.0.67 () {56 vars in 1149 bytes} [Tue May 31 14:26:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:09.219841 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc73302f-9642-4aff-b176-7ba36201d08e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:09.237104 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 449/927] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:09.238421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e01bd1-2f4c-4e99-bad3-8e275eaa76ee None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:09.238741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e01bd1-2f4c-4e99-bad3-8e275eaa76ee None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:09.239012 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e01bd1-2f4c-4e99-bad3-8e275eaa76ee None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:09.240364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 450/928] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:09.240933 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e01bd1-2f4c-4e99-bad3-8e275eaa76ee None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:09.309264 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:09.329403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc73302f-9642-4aff-b176-7ba36201d08e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:09.354529 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e01bd1-2f4c-4e99-bad3-8e275eaa76ee None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:09.363672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499f8b9-406c-450e-b468-b481202ffbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:09.364336 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499f8b9-406c-450e-b468-b481202ffbd3 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:09.364621 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499f8b9-406c-450e-b468-b481202ffbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:09.364900 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499f8b9-406c-450e-b468-b481202ffbd3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:09.365640 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a499f8b9-406c-450e-b468-b481202ffbd3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:09.367051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a499f8b9-406c-450e-b468-b481202ffbd3 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:09.367921 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:09.398675 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 479/929] 10.222.0.67 () {60 vars in 1144 bytes} [Tue May 31 14:26:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:26:09.398675 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53bfd2-dffb-4b97-924a-a33620d2a2c1 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:09.398675 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53bfd2-dffb-4b97-924a-a33620d2a2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:09.398675 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53bfd2-dffb-4b97-924a-a33620d2a2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:09.399139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1e690-b062-477a-afb7-b22bfad9a976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:09.399139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1e690-b062-477a-afb7-b22bfad9a976 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:09.399139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1e690-b062-477a-afb7-b22bfad9a976 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:09.399139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1e690-b062-477a-afb7-b22bfad9a976 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:09.405992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6c53bfd2-dffb-4b97-924a-a33620d2a2c1 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:09.407375 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6c53bfd2-dffb-4b97-924a-a33620d2a2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:09.412995 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b1e690-b062-477a-afb7-b22bfad9a976 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:09.414663 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b1e690-b062-477a-afb7-b22bfad9a976 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:09.436741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-12b1e690-b062-477a-afb7-b22bfad9a976 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:09.436741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:09.436741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:09.436741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:09.436741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:09.436741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:09.436741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:09.436741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:09.436741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:09.436741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:09.436741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:09.436741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:09.436741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:26:09.436741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:26:09.436741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:26:09.437912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:26:09.437912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:09.437912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:09.437912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:09.437912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:09.437912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:09.437912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:09.437912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:09.437912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:09.437912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:09.437912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:09.437912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:09.437912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:09.437912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:09.437912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:09.437912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:09.437912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:09.439385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:26:09.439385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:26:09.439385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:09.439385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:09.442902 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 451/930] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:26:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:09.443749 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1e690-b062-477a-afb7-b22bfad9a976 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:09.490703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 480/931] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:09.490703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c53bfd2-dffb-4b97-924a-a33620d2a2c1 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:09.560438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1e690-b062-477a-afb7-b22bfad9a976 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:09.562728 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:09.602384 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb70e20-3d83-43b1-b15b-05b787b4727a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:09.603372 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb70e20-3d83-43b1-b15b-05b787b4727a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:09.603643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c53bfd2-dffb-4b97-924a-a33620d2a2c1 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:09.603883 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb70e20-3d83-43b1-b15b-05b787b4727a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:09.604263 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb70e20-3d83-43b1-b15b-05b787b4727a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:09.605149 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb70e20-3d83-43b1-b15b-05b787b4727a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:09.606771 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb70e20-3d83-43b1-b15b-05b787b4727a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:09.619755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-aeb70e20-3d83-43b1-b15b-05b787b4727a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 31 14:26:09.619755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:09.619755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:26:09.619755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:26:09.619755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:26:09.619755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:26:09.619755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:26:09.619755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:09.619755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 31 14:26:09.619755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:09.619755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:26:09.619755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:09.619755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:09.619755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:09.619755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:09.619755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:09.619755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:09.620857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:09.620857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:09.620857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:09.620857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:09.620857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:09.620857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:09.620857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:26:09.620857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:26:09.620857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:26:09.620857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:26:09.620857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:09.620857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:09.620857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:09.620857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:09.620857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:09.620857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:09.621866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:09.621866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:09.621866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:09.621866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:09.621866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:09.621866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:09.621866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:09.621866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:09.621866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:09.621866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:09.621866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:26:09.621866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:09.621866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 31 14:26:09.621866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:09.622760 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 452/932] 10.222.0.67 () {56 vars in 1126 bytes} [Tue May 31 14:26:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:09.623401 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb70e20-3d83-43b1-b15b-05b787b4727a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:09.651023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983b5ee-9e65-4614-8940-1410753c4f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:09.651699 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983b5ee-9e65-4614-8940-1410753c4f7d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:09.651980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983b5ee-9e65-4614-8940-1410753c4f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:09.652254 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983b5ee-9e65-4614-8940-1410753c4f7d None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:09.652988 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983b5ee-9e65-4614-8940-1410753c4f7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:09.653428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983b5ee-9e65-4614-8940-1410753c4f7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:09.654707 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983b5ee-9e65-4614-8940-1410753c4f7d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:09.656691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:09.673707 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 481/933] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:09.675251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983b5ee-9e65-4614-8940-1410753c4f7d None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:09.743877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb70e20-3d83-43b1-b15b-05b787b4727a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:09.778147 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe79a1-eb42-4260-b07b-55585f4541bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:09.779310 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe79a1-eb42-4260-b07b-55585f4541bc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:09.779592 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe79a1-eb42-4260-b07b-55585f4541bc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:09.779866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe79a1-eb42-4260-b07b-55585f4541bc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:09.780522 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe79a1-eb42-4260-b07b-55585f4541bc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:09.781879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe79a1-eb42-4260-b07b-55585f4541bc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:09.787594 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983b5ee-9e65-4614-8940-1410753c4f7d None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:09.793953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-33fe79a1-eb42-4260-b07b-55585f4541bc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 31 14:26:09.793953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:09.793953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:26:09.793953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:26:09.793953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:26:09.793953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:26:09.793953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:26:09.793953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:09.793953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 31 14:26:09.793953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:09.793953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:26:09.793953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:09.793953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:09.793953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:09.793953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:09.793953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:09.795163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:09.795163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:09.795163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:09.795163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:09.795163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:09.795163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:09.795163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:09.795163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:26:09.795163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:26:09.795163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:26:09.795163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:26:09.795163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:09.795163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:09.795163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:09.795163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:09.795163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:09.796236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:09.796236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:09.796236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:09.796236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:09.796236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:09.796236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:09.796236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:09.796236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:09.796236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:09.796236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:09.796236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:09.796236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:26:09.796236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:09.796236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 31 14:26:09.796236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:09.798959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 453/934] 10.222.0.67 () {56 vars in 1132 bytes} [Tue May 31 14:26:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:09.800092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe79a1-eb42-4260-b07b-55585f4541bc None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:09.818344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:09.827757 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3e6a4-dc7f-44b4-94ba-0ddfb987391e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:09.828588 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3e6a4-dc7f-44b4-94ba-0ddfb987391e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:09.828876 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3e6a4-dc7f-44b4-94ba-0ddfb987391e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:09.829152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3e6a4-dc7f-44b4-94ba-0ddfb987391e None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:09.829866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3e6a4-dc7f-44b4-94ba-0ddfb987391e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:09.830177 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3e6a4-dc7f-44b4-94ba-0ddfb987391e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:09.831581 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3e6a4-dc7f-44b4-94ba-0ddfb987391e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:09.836639 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 482/935] 10.222.0.67 () {56 vars in 1122 bytes} [Tue May 31 14:26:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:09.837406 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3e6a4-dc7f-44b4-94ba-0ddfb987391e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:09.902458 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:09.916401 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe79a1-eb42-4260-b07b-55585f4541bc None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:09.946034 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3e6a4-dc7f-44b4-94ba-0ddfb987391e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:09.951749 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23bf15-b13c-4215-bec0-74e02c4f2469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:09.952617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23bf15-b13c-4215-bec0-74e02c4f2469 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:09.952970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23bf15-b13c-4215-bec0-74e02c4f2469 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:09.953343 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23bf15-b13c-4215-bec0-74e02c4f2469 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:09.960239 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23bf15-b13c-4215-bec0-74e02c4f2469 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:09.962237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23bf15-b13c-4215-bec0-74e02c4f2469 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:09.977708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-2e23bf15-b13c-4215-bec0-74e02c4f2469 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:26:09.977708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:09.977708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:09.977708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:09.977708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:09.977708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:09.977708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:09.977708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:09.977708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:09.977708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:09.977708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:09.977708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:09.977708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:26:09.977708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:26:09.977708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:26:09.979257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:26:09.979257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:09.979257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:09.979257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:09.979257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:09.979257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:09.979257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:09.979257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:09.979257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:09.979257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:09.979257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:09.979257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:09.979257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:09.979257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:09.979257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:09.979257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:09.979257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:09.980164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:26:09.980164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:26:09.980164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:26:09.980164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:26:09.980164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:26:09.980164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:26:09.980164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:26:09.980164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:09.980164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 454/936] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:26:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:09.981114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23bf15-b13c-4215-bec0-74e02c4f2469 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:09.990782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb53ad2-8bdc-4c5b-b31f-d557c5d2fcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:09.991460 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb53ad2-8bdc-4c5b-b31f-d557c5d2fcd7 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:09.991868 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb53ad2-8bdc-4c5b-b31f-d557c5d2fcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:09.992153 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb53ad2-8bdc-4c5b-b31f-d557c5d2fcd7 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:09.993026 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb53ad2-8bdc-4c5b-b31f-d557c5d2fcd7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:09.993470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb53ad2-8bdc-4c5b-b31f-d557c5d2fcd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:09.995395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb53ad2-8bdc-4c5b-b31f-d557c5d2fcd7 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:10.005357 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 483/937] 10.222.0.67 () {56 vars in 1126 bytes} [Tue May 31 14:26:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:10.005357 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb53ad2-8bdc-4c5b-b31f-d557c5d2fcd7 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:10.062463 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:10.087322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23bf15-b13c-4215-bec0-74e02c4f2469 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:10.110392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb53ad2-8bdc-4c5b-b31f-d557c5d2fcd7 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:10.131158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae4509-f004-4a2c-9c95-e689d4173a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:10.131839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae4509-f004-4a2c-9c95-e689d4173a73 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:10.132130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae4509-f004-4a2c-9c95-e689d4173a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:10.132409 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae4509-f004-4a2c-9c95-e689d4173a73 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:10.133117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae4509-f004-4a2c-9c95-e689d4173a73 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:10.133427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae4509-f004-4a2c-9c95-e689d4173a73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:10.134920 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae4509-f004-4a2c-9c95-e689d4173a73 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:10.140119 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 455/938] 10.222.0.67 () {56 vars in 1122 bytes} [Tue May 31 14:26:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:10.140894 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae4509-f004-4a2c-9c95-e689d4173a73 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:10.157797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da57837-b919-4efd-996e-48e363b2942a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:10.158000 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:10.158842 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da57837-b919-4efd-996e-48e363b2942a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:10.159125 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da57837-b919-4efd-996e-48e363b2942a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:10.159401 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da57837-b919-4efd-996e-48e363b2942a None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:10.160220 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da57837-b919-4efd-996e-48e363b2942a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:10.160525 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da57837-b919-4efd-996e-48e363b2942a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:10.162097 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da57837-b919-4efd-996e-48e363b2942a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:10.175379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 484/939] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:26:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:10.180243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da57837-b919-4efd-996e-48e363b2942a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:10.254030 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae4509-f004-4a2c-9c95-e689d4173a73 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:10.282506 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da57837-b919-4efd-996e-48e363b2942a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:10.291755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ffe3cd-e6d8-4494-b9bb-dd42d96a9197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:10.292681 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ffe3cd-e6d8-4494-b9bb-dd42d96a9197 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:10.293034 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ffe3cd-e6d8-4494-b9bb-dd42d96a9197 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:10.293395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ffe3cd-e6d8-4494-b9bb-dd42d96a9197 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:10.294110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ffe3cd-e6d8-4494-b9bb-dd42d96a9197 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:10.294425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ffe3cd-e6d8-4494-b9bb-dd42d96a9197 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:10.295837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ffe3cd-e6d8-4494-b9bb-dd42d96a9197 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:10.300443 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:10.305343 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 456/940] 10.222.0.67 () {56 vars in 1126 bytes} [Tue May 31 14:26:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:10.305343 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ffe3cd-e6d8-4494-b9bb-dd42d96a9197 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:10.327175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2bc93-f2cb-479c-bab0-cf02e8280169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:10.327834 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2bc93-f2cb-479c-bab0-cf02e8280169 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:10.328125 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2bc93-f2cb-479c-bab0-cf02e8280169 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:10.328406 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2bc93-f2cb-479c-bab0-cf02e8280169 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:10.329203 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2bc93-f2cb-479c-bab0-cf02e8280169 None admin] RBAC: Adding query filter params (interface=public, service_id=c1e59e1d999c4b53870423df786441a3, region_id=RegionOne) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:10.329526 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2bc93-f2cb-479c-bab0-cf02e8280169 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:10.330916 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2bc93-f2cb-479c-bab0-cf02e8280169 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:10.336237 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 485/941] 10.222.0.67 () {56 vars in 1251 bytes} [Tue May 31 14:26:10 2022] GET /identity/v3/endpoints?service_id=c1e59e1d999c4b53870423df786441a3&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:10.337265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2bc93-f2cb-479c-bab0-cf02e8280169 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:10.391183 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:10.417482 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ffe3cd-e6d8-4494-b9bb-dd42d96a9197 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:10.438068 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2bc93-f2cb-479c-bab0-cf02e8280169 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:10.453103 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f2b6d-7b7b-4e1a-83d0-7c7f0bc4809e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:10.454003 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f2b6d-7b7b-4e1a-83d0-7c7f0bc4809e None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:10.454289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f2b6d-7b7b-4e1a-83d0-7c7f0bc4809e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:10.454572 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f2b6d-7b7b-4e1a-83d0-7c7f0bc4809e None admin] PATH_INFO: `/v3/projects/f965a93a2f674598890bd8cafededb2c/users/0b7d100f2407432381d119c25814fff3/roles/8de91fffa27842109896bf18045db796` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:10.456843 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f2b6d-7b7b-4e1a-83d0-7c7f0bc4809e None admin] RBAC: Authorizing `identity:create_grant(project_id=f965a93a2f674598890bd8cafededb2c, user_id=0b7d100f2407432381d119c25814fff3, role_id=8de91fffa27842109896bf18045db796)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:10.458291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f2b6d-7b7b-4e1a-83d0-7c7f0bc4809e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:10.472864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 457/942] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:26:10 2022] PUT /identity/v3/projects/f965a93a2f674598890bd8cafededb2c/users/0b7d100f2407432381d119c25814fff3/roles/8de91fffa27842109896bf18045db796 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:26:10.474003 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f2b6d-7b7b-4e1a-83d0-7c7f0bc4809e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:10.485876 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de3349-0bc2-40c6-b0d9-a07555277209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:10.486628 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de3349-0bc2-40c6-b0d9-a07555277209 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:10.486995 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de3349-0bc2-40c6-b0d9-a07555277209 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:10.487278 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de3349-0bc2-40c6-b0d9-a07555277209 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:10.488136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de3349-0bc2-40c6-b0d9-a07555277209 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:10.489682 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de3349-0bc2-40c6-b0d9-a07555277209 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:10.519048 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 486/943] 10.222.0.67 () {60 vars in 1144 bytes} [Tue May 31 14:26:10 2022] POST /identity/v3/endpoints => generated 347 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:26:10.612856 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f2b6d-7b7b-4e1a-83d0-7c7f0bc4809e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:10.656337 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:10.663934 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd8ed6-701d-4354-83c0-a6672317aa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:10.665067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd8ed6-701d-4354-83c0-a6672317aa6a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:10.665356 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd8ed6-701d-4354-83c0-a6672317aa6a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:10.665641 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd8ed6-701d-4354-83c0-a6672317aa6a None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:10.666971 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd8ed6-701d-4354-83c0-a6672317aa6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:10.668708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd8ed6-701d-4354-83c0-a6672317aa6a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:10.676697 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 458/944] 10.222.0.67 () {56 vars in 1087 bytes} [Tue May 31 14:26:10 2022] GET /identity/v3/services => generated 1497 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:26:11.784222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47bce9-9fee-49a5-8d16-0f10746025ab None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:11.784222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47bce9-9fee-49a5-8d16-0f10746025ab None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:11.784222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47bce9-9fee-49a5-8d16-0f10746025ab None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:11.785750 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 487/945] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:11.789513 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bf4ee-9b12-4cc0-9884-3c24248c4507 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:11.790115 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bf4ee-9b12-4cc0-9884-3c24248c4507 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:11.790115 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bf4ee-9b12-4cc0-9884-3c24248c4507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:11.820827 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-968bf4ee-9b12-4cc0-9884-3c24248c4507 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:11.822180 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-968bf4ee-9b12-4cc0-9884-3c24248c4507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:11.918287 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 459/946] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:11.941419 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed47bce9-9fee-49a5-8d16-0f10746025ab None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:11.980209 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:12.050639 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed47bce9-9fee-49a5-8d16-0f10746025ab None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:12.086397 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980409ce-5410-4f4c-9585-27c335f350ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:12.087465 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980409ce-5410-4f4c-9585-27c335f350ce None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:12.088636 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980409ce-5410-4f4c-9585-27c335f350ce None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:12.089120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980409ce-5410-4f4c-9585-27c335f350ce None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:12.090046 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980409ce-5410-4f4c-9585-27c335f350ce None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:12.091715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980409ce-5410-4f4c-9585-27c335f350ce None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:12.093299 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: WARNING keystone.server.flask.common [None req-980409ce-5410-4f4c-9585-27c335f350ce None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 31 14:26:12.113582 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.sql.core [None req-980409ce-5410-4f4c-9585-27c335f350ce None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 31 14:26:12.113582 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 31 14:26:12.113582 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [parameters: {'id': '1bcff7c4592d4fd9962a4495aabda911', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 31 14:26:12.113582 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80234) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 31 14:26:12.131992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-980409ce-5410-4f4c-9585-27c335f350ce None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1bcff7c4592d4fd9962a4495aabda911.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1bcff7c4592d4fd9962a4495aabda911. May 31 14:26:12.131992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:12.131992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 31 14:26:12.131992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self.dialect.do_execute( May 31 14:26:12.131992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 31 14:26:12.131992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 31 14:26:12.131992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 31 14:26:12.131992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application result = self._query(query) May 31 14:26:12.131992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 31 14:26:12.131992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application conn.query(q) May 31 14:26:12.131992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 31 14:26:12.131992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 31 14:26:12.131992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 31 14:26:12.132762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application result.read() May 31 14:26:12.132762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 31 14:26:12.132762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 31 14:26:12.132762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 31 14:26:12.132762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application packet.raise_for_error() May 31 14:26:12.132762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 31 14:26:12.132762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 31 14:26:12.132762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 31 14:26:12.132762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 31 14:26:12.132762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 31 14:26:12.132762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:12.132762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 31 14:26:12.132762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:12.132762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:12.132762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 31 14:26:12.132762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 31 14:26:12.132762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 31 14:26:12.132762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return project_ref.to_dict() May 31 14:26:12.133726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 31 14:26:12.133726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application next(self.gen) May 31 14:26:12.133726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 31 14:26:12.133726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application yield resource May 31 14:26:12.133726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 31 14:26:12.133726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application next(self.gen) May 31 14:26:12.133726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 31 14:26:12.133726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self.session.rollback() May 31 14:26:12.133726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 31 14:26:12.133726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self.force_reraise() May 31 14:26:12.133726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 31 14:26:12.133726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise self.value May 31 14:26:12.133726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 31 14:26:12.133726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 31 14:26:12.133726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 31 14:26:12.133726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application session.commit() May 31 14:26:12.133726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 31 14:26:12.134667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 31 14:26:12.134667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 31 14:26:12.134667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._prepare_impl() May 31 14:26:12.134667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 31 14:26:12.134667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self.session.flush() May 31 14:26:12.134667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 31 14:26:12.134667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._flush(objects) May 31 14:26:12.134667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 31 14:26:12.134667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 31 14:26:12.134667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 31 14:26:12.134667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application compat.raise_( May 31 14:26:12.134667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 31 14:26:12.134667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception May 31 14:26:12.134667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 31 14:26:12.134667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application flush_context.execute() May 31 14:26:12.134667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 31 14:26:12.134667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rec.execute(self) May 31 14:26:12.135660 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 31 14:26:12.135660 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 31 14:26:12.135660 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 31 14:26:12.135660 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application _emit_insert_statements( May 31 14:26:12.135660 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 31 14:26:12.135660 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application c = connection._execute_20( May 31 14:26:12.135660 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 31 14:26:12.135660 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 31 14:26:12.135660 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 31 14:26:12.135660 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 31 14:26:12.135660 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 31 14:26:12.135660 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ret = self._execute_context( May 31 14:26:12.135660 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 31 14:26:12.135660 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 31 14:26:12.135660 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 31 14:26:12.136483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 31 14:26:12.136483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 31 14:26:12.136483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception May 31 14:26:12.136483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 31 14:26:12.136483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self.dialect.do_execute( May 31 14:26:12.136483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 31 14:26:12.136483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 31 14:26:12.136483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 31 14:26:12.136483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application result = self._query(query) May 31 14:26:12.136483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 31 14:26:12.136483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application conn.query(q) May 31 14:26:12.136483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 31 14:26:12.136483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 31 14:26:12.136483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 31 14:26:12.136483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application result.read() May 31 14:26:12.136483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 31 14:26:12.136483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 31 14:26:12.137365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 31 14:26:12.137365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application packet.raise_for_error() May 31 14:26:12.137365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 31 14:26:12.137365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 31 14:26:12.137365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 31 14:26:12.137365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 31 14:26:12.137365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 31 14:26:12.137365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 31 14:26:12.137365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [parameters: {'id': '1bcff7c4592d4fd9962a4495aabda911', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 31 14:26:12.137365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 31 14:26:12.137365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:12.137365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:26:12.137365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:12.137365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:12.137365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 31 14:26:12.138175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 31 14:26:12.138175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 31 14:26:12.138175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 31 14:26:12.138175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 31 14:26:12.138175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:12.138175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:26:12.138175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:12.138175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:12.138175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:12.138175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:12.138175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:12.138175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:12.138175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:12.138175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:12.138175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:12.138175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:12.138175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:12.145076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:12.145076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 31 14:26:12.145076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 31 14:26:12.145076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:12.145076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:12.145076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 31 14:26:12.145076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.Conflict( May 31 14:26:12.145076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1bcff7c4592d4fd9962a4495aabda911. May 31 14:26:12.145076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:12.145076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 488/947] 10.222.0.67 () {60 vars in 1140 bytes} [Tue May 31 14:26:11 2022] POST /identity/v3/projects => generated 262 bytes in 192 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:12.150845 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bf4ee-9b12-4cc0-9884-3c24248c4507 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:12.258723 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bf4ee-9b12-4cc0-9884-3c24248c4507 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:12.285880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:12.294553 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea408854-7530-46be-bb73-baaea0f4f032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:12.295418 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea408854-7530-46be-bb73-baaea0f4f032 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:12.295782 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea408854-7530-46be-bb73-baaea0f4f032 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:12.296261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea408854-7530-46be-bb73-baaea0f4f032 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:12.303570 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea408854-7530-46be-bb73-baaea0f4f032 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:12.304966 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea408854-7530-46be-bb73-baaea0f4f032 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:12.318886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-ea408854-7530-46be-bb73-baaea0f4f032 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:26:12.318886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:12.318886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:12.318886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:12.318886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:12.318886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:12.318886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:12.318886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:12.318886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:12.318886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:12.318886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:12.318886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:12.318886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:26:12.318886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:26:12.318886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:26:12.319770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:26:12.319770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:12.319770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:12.319770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:12.319770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:12.319770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:12.319770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:12.319770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:12.319770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:12.319770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:12.319770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:12.319770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:12.319770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:12.319770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:12.319770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:12.319770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:12.319770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:12.320691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:26:12.320691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:26:12.320691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:26:12.320691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:26:12.320691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:26:12.320691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:26:12.320691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:26:12.320691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:12.320691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 460/948] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:26:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:26:12.328349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980409ce-5410-4f4c-9585-27c335f350ce None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:12.422529 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:12.431513 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980409ce-5410-4f4c-9585-27c335f350ce None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:12.467741 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004508e-91ae-4bef-8811-ab73969576c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:12.468415 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004508e-91ae-4bef-8811-ab73969576c4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:12.468697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004508e-91ae-4bef-8811-ab73969576c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:12.468971 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004508e-91ae-4bef-8811-ab73969576c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:12.469740 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3004508e-91ae-4bef-8811-ab73969576c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:12.470048 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3004508e-91ae-4bef-8811-ab73969576c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:12.471724 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3004508e-91ae-4bef-8811-ab73969576c4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:12.495684 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 489/949] 10.222.0.67 () {56 vars in 1112 bytes} [Tue May 31 14:26:12 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:12.961093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d7f29-a92d-4ea5-9e10-dc913cce89bd None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:12.961650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d7f29-a92d-4ea5-9e10-dc913cce89bd None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:12.961927 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d7f29-a92d-4ea5-9e10-dc913cce89bd None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:12.963721 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 461/950] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:12.968998 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02058d9d-0bd4-4aa7-8d9f-ee54559bd631 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:12.969404 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02058d9d-0bd4-4aa7-8d9f-ee54559bd631 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:12.969734 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02058d9d-0bd4-4aa7-8d9f-ee54559bd631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:12.987663 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-02058d9d-0bd4-4aa7-8d9f-ee54559bd631 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:12.989170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-02058d9d-0bd4-4aa7-8d9f-ee54559bd631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:13.060332 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 490/951] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:13.082966 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d7f29-a92d-4ea5-9e10-dc913cce89bd None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:13.155803 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7165ae-ac06-4467-abfa-4f312c43bd1d None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:13.156114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7165ae-ac06-4467-abfa-4f312c43bd1d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:13.156375 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7165ae-ac06-4467-abfa-4f312c43bd1d None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:13.158178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 491/952] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:13.160534 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c32e0-0ebb-4fbe-a77d-0468197b6e05 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:13.160833 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c32e0-0ebb-4fbe-a77d-0468197b6e05 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:13.161119 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c32e0-0ebb-4fbe-a77d-0468197b6e05 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:13.162588 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 492/953] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:13.163745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6b91a-9e92-49b3-bc0c-855d5032eb0b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:13.164035 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6b91a-9e92-49b3-bc0c-855d5032eb0b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:13.164301 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6b91a-9e92-49b3-bc0c-855d5032eb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:13.185472 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6fc6b91a-9e92-49b3-bc0c-855d5032eb0b None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:13.186782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6fc6b91a-9e92-49b3-bc0c-855d5032eb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:13.198999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d7f29-a92d-4ea5-9e10-dc913cce89bd None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:13.236584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395204d-cee9-4641-83e7-50b994ab8cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:13.238020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395204d-cee9-4641-83e7-50b994ab8cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:13.238564 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395204d-cee9-4641-83e7-50b994ab8cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:13.239025 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395204d-cee9-4641-83e7-50b994ab8cd8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:13.251477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395204d-cee9-4641-83e7-50b994ab8cd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:26:13.256595 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 493/954] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 31 14:26:13.257891 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bed83-e93e-4e13-95d7-f3aec7c38d1e None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:13.258183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bed83-e93e-4e13-95d7-f3aec7c38d1e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:13.258466 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bed83-e93e-4e13-95d7-f3aec7c38d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:13.260581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-7395204d-cee9-4641-83e7-50b994ab8cd8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 31 14:26:13.260581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:13.260581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:13.260581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:13.260581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:13.260581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:13.260581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:13.260581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:13.260581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:13.260581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:13.260581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:13.260581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:13.260581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:26:13.260581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:26:13.260581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:26:13.260581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:26:13.261600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:26:13.261600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:26:13.261600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:26:13.261600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:26:13.261600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:13.261600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:13.261600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:13.261600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:13.261600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:13.261600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:13.261600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:13.261600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:13.261600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:13.261600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:13.261600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:13.261600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:13.262483 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:13.262483 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:13.262483 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:13.262483 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:13.262483 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:26:13.262483 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:26:13.262483 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:26:13.262483 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:26:13.262483 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:26:13.262483 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:26:13.262483 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 31 14:26:13.262483 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:13.266626 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 462/955] 10.222.0.67 () {56 vars in 1120 bytes} [Tue May 31 14:26:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:13.267447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395204d-cee9-4641-83e7-50b994ab8cd8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:13.282530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-df0bed83-e93e-4e13-95d7-f3aec7c38d1e None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:13.283976 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-df0bed83-e93e-4e13-95d7-f3aec7c38d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:13.292940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:13.320559 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:13.357259 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 494/956] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:13.358314 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bed83-e93e-4e13-95d7-f3aec7c38d1e None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:13.387778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395204d-cee9-4641-83e7-50b994ab8cd8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:13.426343 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4534041-916d-4ce5-b007-df9cd2ac5252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:13.427045 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4534041-916d-4ce5-b007-df9cd2ac5252 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:13.427334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4534041-916d-4ce5-b007-df9cd2ac5252 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:13.427754 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4534041-916d-4ce5-b007-df9cd2ac5252 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:13.428362 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4534041-916d-4ce5-b007-df9cd2ac5252 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:13.428672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4534041-916d-4ce5-b007-df9cd2ac5252 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:13.430038 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4534041-916d-4ce5-b007-df9cd2ac5252 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:13.438891 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 463/957] 10.222.0.67 () {56 vars in 1115 bytes} [Tue May 31 14:26:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:13.439683 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4534041-916d-4ce5-b007-df9cd2ac5252 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:13.467177 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bed83-e93e-4e13-95d7-f3aec7c38d1e None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:13.502719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb3cad-a9ac-4fd1-a274-3bbf7369bfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:13.503469 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb3cad-a9ac-4fd1-a274-3bbf7369bfb8 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:13.503753 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb3cad-a9ac-4fd1-a274-3bbf7369bfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:13.504033 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb3cad-a9ac-4fd1-a274-3bbf7369bfb8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:13.504693 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb3cad-a9ac-4fd1-a274-3bbf7369bfb8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:13.506076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb3cad-a9ac-4fd1-a274-3bbf7369bfb8 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:13.517908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-3edb3cad-a9ac-4fd1-a274-3bbf7369bfb8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 31 14:26:13.517908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:13.517908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:26:13.517908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:26:13.517908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:26:13.517908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:26:13.517908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:26:13.517908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:13.517908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 31 14:26:13.517908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:13.517908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:26:13.517908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:13.517908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:13.517908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:13.517908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:13.517908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:13.517908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:13.519036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:13.519036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:13.519036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:13.519036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:13.519036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:13.519036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:13.519036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:26:13.519036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:26:13.519036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:26:13.519036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:26:13.519036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:13.519036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:13.519036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:13.519036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:13.519036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:13.519036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:13.519898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:13.519898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:26:13.519898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:13.519898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:13.519898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:13.519898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:13.519898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:13.519898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:13.519898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:13.519898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:13.519898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:26:13.519898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:13.519898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 31 14:26:13.519898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:13.519898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 495/958] 10.222.0.67 () {56 vars in 1126 bytes} [Tue May 31 14:26:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:13.520803 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb3cad-a9ac-4fd1-a274-3bbf7369bfb8 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:13.533278 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:13.552269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4534041-916d-4ce5-b007-df9cd2ac5252 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:13.562233 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:13.590333 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a21ea2-0484-4d2d-83b3-4a5d990efa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:13.591170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a21ea2-0484-4d2d-83b3-4a5d990efa53 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:13.591438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a21ea2-0484-4d2d-83b3-4a5d990efa53 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:13.591737 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a21ea2-0484-4d2d-83b3-4a5d990efa53 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:13.592356 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a21ea2-0484-4d2d-83b3-4a5d990efa53 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:13.593852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a21ea2-0484-4d2d-83b3-4a5d990efa53 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:13.606691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e5a21ea2-0484-4d2d-83b3-4a5d990efa53 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 31 14:26:13.606691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:13.606691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:26:13.606691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:26:13.606691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:26:13.606691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:26:13.606691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:26:13.606691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:13.606691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 31 14:26:13.606691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:13.606691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:26:13.606691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:13.606691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:13.606691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:13.606691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:13.606691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:13.606691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:13.607741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:13.607741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:13.607741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:13.607741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:13.607741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:13.607741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:13.607741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:26:13.607741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:26:13.607741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:26:13.607741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:26:13.607741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:13.607741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:13.607741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:13.607741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:13.607741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:13.607741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:13.608599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:13.608599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:13.608599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:13.608599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:13.608599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:13.608599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:13.608599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:13.608599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:13.608599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:13.608599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:13.608599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:26:13.608599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:13.608599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 31 14:26:13.608599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:13.608599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 464/959] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:26:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:13.609996 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a21ea2-0484-4d2d-83b3-4a5d990efa53 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:13.630128 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb3cad-a9ac-4fd1-a274-3bbf7369bfb8 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:13.665646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b8e31-0795-4936-9519-78c3297e7ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:13.667016 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b8e31-0795-4936-9519-78c3297e7ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:13.667300 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b8e31-0795-4936-9519-78c3297e7ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:13.667567 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b8e31-0795-4936-9519-78c3297e7ee0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:13.679000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b8e31-0795-4936-9519-78c3297e7ee0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:13.680494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b8e31-0795-4936-9519-78c3297e7ee0 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:13.699065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-385b8e31-0795-4936-9519-78c3297e7ee0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:13.699065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:13.699065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:13.699065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:13.699065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:13.699065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:13.699065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:13.699065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:13.699065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:13.699065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:13.699065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:13.699065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:13.699065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:26:13.699065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:26:13.699065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:26:13.700039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:26:13.700039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:13.700039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:13.700039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:13.700039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:13.700039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:13.700039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:13.700039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:13.700039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:26:13.700039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:13.700039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:13.700039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:13.700039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:13.700039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:13.700039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:13.700039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:13.700039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:13.701844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:26:13.701844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:26:13.701844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:13.701844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:13.701844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 496/960] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:26:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:13.701844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b8e31-0795-4936-9519-78c3297e7ee0 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:13.719482 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a21ea2-0484-4d2d-83b3-4a5d990efa53 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:13.755419 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33a388-66a1-42ec-8c14-e9cbb9f4a52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:13.756023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33a388-66a1-42ec-8c14-e9cbb9f4a52c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:13.756307 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33a388-66a1-42ec-8c14-e9cbb9f4a52c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:13.756588 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33a388-66a1-42ec-8c14-e9cbb9f4a52c None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:13.757303 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33a388-66a1-42ec-8c14-e9cbb9f4a52c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:13.757616 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33a388-66a1-42ec-8c14-e9cbb9f4a52c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:13.759016 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33a388-66a1-42ec-8c14-e9cbb9f4a52c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:13.764497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 465/961] 10.222.0.67 () {56 vars in 1122 bytes} [Tue May 31 14:26:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:13.765302 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33a388-66a1-42ec-8c14-e9cbb9f4a52c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:13.769855 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:13.806295 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:13.816391 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b8e31-0795-4936-9519-78c3297e7ee0 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:13.854348 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706bdef-6f96-4e7e-ba6c-94345ab0c9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:13.855062 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706bdef-6f96-4e7e-ba6c-94345ab0c9ea None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:13.855481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706bdef-6f96-4e7e-ba6c-94345ab0c9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:13.855740 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706bdef-6f96-4e7e-ba6c-94345ab0c9ea None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:13.856569 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706bdef-6f96-4e7e-ba6c-94345ab0c9ea None admin] RBAC: Adding query filter params (name=compute) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:13.856879 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706bdef-6f96-4e7e-ba6c-94345ab0c9ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:13.858254 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706bdef-6f96-4e7e-ba6c-94345ab0c9ea None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:13.868095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 497/962] 10.222.0.67 () {56 vars in 1112 bytes} [Tue May 31 14:26:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:13.868896 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706bdef-6f96-4e7e-ba6c-94345ab0c9ea None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:13.871320 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33a388-66a1-42ec-8c14-e9cbb9f4a52c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:13.909545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb22475-cfed-439e-ae9d-b96bb618cb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:13.910602 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb22475-cfed-439e-ae9d-b96bb618cb7e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:13.910944 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb22475-cfed-439e-ae9d-b96bb618cb7e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:13.911214 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb22475-cfed-439e-ae9d-b96bb618cb7e None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:13.911943 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb22475-cfed-439e-ae9d-b96bb618cb7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:13.912614 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb22475-cfed-439e-ae9d-b96bb618cb7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:13.914468 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb22475-cfed-439e-ae9d-b96bb618cb7e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:13.928281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 466/963] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:13.929079 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb22475-cfed-439e-ae9d-b96bb618cb7e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:13.977632 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706bdef-6f96-4e7e-ba6c-94345ab0c9ea None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:13.999536 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:14.012500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f227e-d004-416e-bd48-87cc17eb1732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:14.013097 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f227e-d004-416e-bd48-87cc17eb1732 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:14.013382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f227e-d004-416e-bd48-87cc17eb1732 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:14.013667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f227e-d004-416e-bd48-87cc17eb1732 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:14.014375 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f227e-d004-416e-bd48-87cc17eb1732 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:14.014708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f227e-d004-416e-bd48-87cc17eb1732 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:14.016095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f227e-d004-416e-bd48-87cc17eb1732 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:14.020844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 498/964] 10.222.0.67 () {56 vars in 1122 bytes} [Tue May 31 14:26:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:14.021608 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f227e-d004-416e-bd48-87cc17eb1732 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:14.036502 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:14.044902 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb22475-cfed-439e-ae9d-b96bb618cb7e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:14.079490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6be2de-381c-4534-ace4-0a6cb097ad18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:14.080159 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6be2de-381c-4534-ace4-0a6cb097ad18 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:14.080443 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6be2de-381c-4534-ace4-0a6cb097ad18 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:14.080724 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6be2de-381c-4534-ace4-0a6cb097ad18 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:14.081434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6be2de-381c-4534-ace4-0a6cb097ad18 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:14.081747 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6be2de-381c-4534-ace4-0a6cb097ad18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:14.083176 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6be2de-381c-4534-ace4-0a6cb097ad18 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:14.087650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 467/965] 10.222.0.67 () {56 vars in 1112 bytes} [Tue May 31 14:26:13 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:14.088922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6be2de-381c-4534-ace4-0a6cb097ad18 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:14.158594 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f227e-d004-416e-bd48-87cc17eb1732 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:14.204400 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5156fb-0b05-4daa-92be-5e2d5f1affb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:14.206016 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5156fb-0b05-4daa-92be-5e2d5f1affb7 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:14.206664 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5156fb-0b05-4daa-92be-5e2d5f1affb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:14.207206 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5156fb-0b05-4daa-92be-5e2d5f1affb7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:14.219464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5156fb-0b05-4daa-92be-5e2d5f1affb7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:14.221818 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5156fb-0b05-4daa-92be-5e2d5f1affb7 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:14.236371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6be2de-381c-4534-ace4-0a6cb097ad18 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:14.240349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-5b5156fb-0b05-4daa-92be-5e2d5f1affb7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 31 14:26:14.240349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:14.240349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:14.240349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:14.240349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:14.240349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:14.240349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:14.240349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:14.240349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:14.240349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:14.240349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:14.240349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:14.240349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:26:14.240349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:26:14.240349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:26:14.240349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:26:14.241578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:14.241578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:14.241578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:26:14.241578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:26:14.241578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:26:14.241578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:26:14.241578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:14.241578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:14.241578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:14.241578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:14.241578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:14.241578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:26:14.241578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:14.241578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:14.241578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:14.241578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:14.241578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:14.241578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:14.242785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:14.242785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:14.242785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:26:14.242785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:26:14.242785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:26:14.242785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:26:14.242785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:26:14.242785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:26:14.242785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 31 14:26:14.242785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:14.249408 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 499/966] 10.222.0.67 () {56 vars in 1117 bytes} [Tue May 31 14:26:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:14.251132 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5156fb-0b05-4daa-92be-5e2d5f1affb7 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:14.291022 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242f3d4-c4ff-4cda-8bda-aee8a7d10c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:14.292103 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242f3d4-c4ff-4cda-8bda-aee8a7d10c20 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:14.292442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242f3d4-c4ff-4cda-8bda-aee8a7d10c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:14.292728 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242f3d4-c4ff-4cda-8bda-aee8a7d10c20 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:14.293774 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242f3d4-c4ff-4cda-8bda-aee8a7d10c20 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:14.296028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242f3d4-c4ff-4cda-8bda-aee8a7d10c20 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:14.319465 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:14.323927 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 468/967] 10.222.0.67 () {60 vars in 1144 bytes} [Tue May 31 14:26:14 2022] POST /identity/v3/endpoints => generated 349 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:26:14.391123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5156fb-0b05-4daa-92be-5e2d5f1affb7 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:14.438283 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4e519-0248-4c38-a455-4dd1052d7d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:14.439013 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4e519-0248-4c38-a455-4dd1052d7d31 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:14.439296 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4e519-0248-4c38-a455-4dd1052d7d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:14.439576 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4e519-0248-4c38-a455-4dd1052d7d31 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:14.440429 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4e519-0248-4c38-a455-4dd1052d7d31 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:14.440739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4e519-0248-4c38-a455-4dd1052d7d31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:14.442277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4e519-0248-4c38-a455-4dd1052d7d31 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:14.464011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 500/968] 10.222.0.67 () {56 vars in 1149 bytes} [Tue May 31 14:26:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:14.471961 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242f3d4-c4ff-4cda-8bda-aee8a7d10c20 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:14.494011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:14.591934 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242f3d4-c4ff-4cda-8bda-aee8a7d10c20 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:14.627700 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad3ba4-0d00-4bf2-8eec-1604dd014e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:14.628708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad3ba4-0d00-4bf2-8eec-1604dd014e44 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:14.629202 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad3ba4-0d00-4bf2-8eec-1604dd014e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:14.629726 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad3ba4-0d00-4bf2-8eec-1604dd014e44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:14.701010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad3ba4-0d00-4bf2-8eec-1604dd014e44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:14.702860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad3ba4-0d00-4bf2-8eec-1604dd014e44 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:14.721231 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:14.807718 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 469/969] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:26:14 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 31 14:26:14.813675 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4e519-0248-4c38-a455-4dd1052d7d31 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:14.840497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09195e54-232c-4fd1-85ce-2496fb212fe4 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:14.840828 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09195e54-232c-4fd1-85ce-2496fb212fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:14.841032 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09195e54-232c-4fd1-85ce-2496fb212fe4 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:14.842730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 470/970] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 31 14:26:14.847570 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ccc02-8adc-47dd-a807-13d787ffe365 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:14.847863 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ccc02-8adc-47dd-a807-13d787ffe365 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:14.848142 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ccc02-8adc-47dd-a807-13d787ffe365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:14.873628 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-bb6ccc02-8adc-47dd-a807-13d787ffe365 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:14.875416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bb6ccc02-8adc-47dd-a807-13d787ffe365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:14.934861 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:14.940237 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4e519-0248-4c38-a455-4dd1052d7d31 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:14.970701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 471/971] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 31 14:26:14.986931 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6973537-90c0-4761-959a-37e6a75ad112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:14.987736 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6973537-90c0-4761-959a-37e6a75ad112 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:14.988021 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6973537-90c0-4761-959a-37e6a75ad112 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:14.988329 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6973537-90c0-4761-959a-37e6a75ad112 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:14.995085 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ccc02-8adc-47dd-a807-13d787ffe365 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:14.999339 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6973537-90c0-4761-959a-37e6a75ad112 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:15.000795 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6973537-90c0-4761-959a-37e6a75ad112 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:15.018698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-f6973537-90c0-4761-959a-37e6a75ad112 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:15.018698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:15.018698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:15.018698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:15.018698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:15.018698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:15.018698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:15.018698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:15.018698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:15.018698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:15.018698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:15.018698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:15.018698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:26:15.018698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:26:15.018698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:26:15.019607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:26:15.019607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:15.019607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:15.019607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:15.019607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:15.019607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:15.019607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:15.019607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:15.019607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:26:15.019607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:15.019607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:15.019607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:15.019607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:15.019607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:15.019607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:15.019607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:15.019607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:15.020497 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:26:15.020497 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:26:15.020497 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:15.020497 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:15.020497 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 501/972] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:26:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:15.027619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6973537-90c0-4761-959a-37e6a75ad112 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:15.052453 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:15.111185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ccc02-8adc-47dd-a807-13d787ffe365 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:15.141509 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6973537-90c0-4761-959a-37e6a75ad112 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:15.147879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa4f8b-9ce4-46c1-96d6-a52d8602cc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:15.148810 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa4f8b-9ce4-46c1-96d6-a52d8602cc69 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:15.149304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa4f8b-9ce4-46c1-96d6-a52d8602cc69 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:15.149786 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa4f8b-9ce4-46c1-96d6-a52d8602cc69 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:15.151267 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa4f8b-9ce4-46c1-96d6-a52d8602cc69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:15.152905 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa4f8b-9ce4-46c1-96d6-a52d8602cc69 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:15.154431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: WARNING keystone.server.flask.common [None req-61fa4f8b-9ce4-46c1-96d6-a52d8602cc69 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 31 14:26:15.168096 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.sql.core [None req-61fa4f8b-9ce4-46c1-96d6-a52d8602cc69 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 31 14:26:15.168096 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 31 14:26:15.168096 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [parameters: {'user_id': 'd6798d35dcf740739190b47604ccc3de', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 31 14:26:15.168096 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80233) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 31 14:26:15.177423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-61fa4f8b-9ce4-46c1-96d6-a52d8602cc69 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 31 14:26:15.177423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:15.177423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 31 14:26:15.177423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.dialect.do_execute( May 31 14:26:15.177423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 31 14:26:15.177423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 31 14:26:15.177423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 31 14:26:15.177423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result = self._query(query) May 31 14:26:15.177423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 31 14:26:15.177423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application conn.query(q) May 31 14:26:15.177423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 31 14:26:15.177423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 31 14:26:15.177423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 31 14:26:15.177423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result.read() May 31 14:26:15.177423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 31 14:26:15.178284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 31 14:26:15.178284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 31 14:26:15.178284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application packet.raise_for_error() May 31 14:26:15.178284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 31 14:26:15.178284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 31 14:26:15.178284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 31 14:26:15.178284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 31 14:26:15.178284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 31 14:26:15.178284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:15.178284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 31 14:26:15.178284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:15.178284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:15.178284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 31 14:26:15.178284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 31 14:26:15.178284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 31 14:26:15.178284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 31 14:26:15.178284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 31 14:26:15.178284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application next(self.gen) May 31 14:26:15.179332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 31 14:26:15.179332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application yield resource May 31 14:26:15.179332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 31 14:26:15.179332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application next(self.gen) May 31 14:26:15.179332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 31 14:26:15.179332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.session.rollback() May 31 14:26:15.179332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 31 14:26:15.179332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.force_reraise() May 31 14:26:15.179332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 31 14:26:15.179332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise self.value May 31 14:26:15.179332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 31 14:26:15.179332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 31 14:26:15.179332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 31 14:26:15.179332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application session.commit() May 31 14:26:15.179332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 31 14:26:15.179332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 31 14:26:15.179332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 31 14:26:15.180364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._prepare_impl() May 31 14:26:15.180364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 31 14:26:15.180364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.session.flush() May 31 14:26:15.180364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 31 14:26:15.180364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._flush(objects) May 31 14:26:15.180364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 31 14:26:15.180364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 31 14:26:15.180364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 31 14:26:15.180364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application compat.raise_( May 31 14:26:15.180364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 31 14:26:15.180364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception May 31 14:26:15.180364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 31 14:26:15.180364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application flush_context.execute() May 31 14:26:15.180364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 31 14:26:15.180364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rec.execute(self) May 31 14:26:15.180364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 31 14:26:15.180364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 31 14:26:15.182097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 31 14:26:15.182097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application _emit_insert_statements( May 31 14:26:15.182097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 31 14:26:15.182097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result = connection._execute_20( May 31 14:26:15.182097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 31 14:26:15.182097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 31 14:26:15.182097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 31 14:26:15.182097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 31 14:26:15.182097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 31 14:26:15.182097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ret = self._execute_context( May 31 14:26:15.182097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 31 14:26:15.182097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 31 14:26:15.182097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 31 14:26:15.182097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 31 14:26:15.182097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 31 14:26:15.182097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception May 31 14:26:15.183633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 31 14:26:15.183633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.dialect.do_execute( May 31 14:26:15.183633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 31 14:26:15.183633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 31 14:26:15.183633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 31 14:26:15.183633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result = self._query(query) May 31 14:26:15.183633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 31 14:26:15.183633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application conn.query(q) May 31 14:26:15.183633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 31 14:26:15.183633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 31 14:26:15.183633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 31 14:26:15.183633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result.read() May 31 14:26:15.183633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 31 14:26:15.183633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 31 14:26:15.183633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 31 14:26:15.183633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application packet.raise_for_error() May 31 14:26:15.185907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 31 14:26:15.185907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 31 14:26:15.185907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 31 14:26:15.185907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 31 14:26:15.185907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 31 14:26:15.185907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 31 14:26:15.185907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd6798d35dcf740739190b47604ccc3de', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 31 14:26:15.185907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 31 14:26:15.185907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:15.185907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:26:15.185907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:15.185907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:15.185907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:15.185907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:15.185907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:15.187912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:15.187912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:15.187912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:15.187912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:15.187912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:15.187912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:15.187912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:15.187912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 31 14:26:15.187912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 31 14:26:15.187912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:15.187912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:15.187912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:26:15.187912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:26:15.187912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:26:15.187912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:26:15.187912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 31 14:26:15.187912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 31 14:26:15.190246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 31 14:26:15.190246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 31 14:26:15.190246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 31 14:26:15.190246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 31 14:26:15.190246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 31 14:26:15.190246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:15.190246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 472/973] 10.222.0.67 () {60 vars in 1131 bytes} [Tue May 31 14:26:14 2022] POST /identity/v3/users => generated 160 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 31 14:26:15.191205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5b0ea-6e01-4440-8302-b05c11536fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:15.191205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5b0ea-6e01-4440-8302-b05c11536fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:15.191205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5b0ea-6e01-4440-8302-b05c11536fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:15.191205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5b0ea-6e01-4440-8302-b05c11536fe6 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:15.192014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f5b0ea-6e01-4440-8302-b05c11536fe6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:15.192014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f5b0ea-6e01-4440-8302-b05c11536fe6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:15.192014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f5b0ea-6e01-4440-8302-b05c11536fe6 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:15.192494 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa4f8b-9ce4-46c1-96d6-a52d8602cc69 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:15.207826 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 502/974] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:26:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:15.208733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5b0ea-6e01-4440-8302-b05c11536fe6 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:15.224676 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:15.312285 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa4f8b-9ce4-46c1-96d6-a52d8602cc69 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:15.315867 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5b0ea-6e01-4440-8302-b05c11536fe6 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:15.323510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:15.351954 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdaf354-f8f2-491a-bf52-77d70e7031b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:15.352766 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaf354-f8f2-491a-bf52-77d70e7031b2 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:15.353046 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaf354-f8f2-491a-bf52-77d70e7031b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:15.353335 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaf354-f8f2-491a-bf52-77d70e7031b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:15.356355 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea90c2-aedd-4c6d-83e1-a0a726831c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:15.357411 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea90c2-aedd-4c6d-83e1-a0a726831c09 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:15.357919 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea90c2-aedd-4c6d-83e1-a0a726831c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:15.358399 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea90c2-aedd-4c6d-83e1-a0a726831c09 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:15.360774 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdaf354-f8f2-491a-bf52-77d70e7031b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:15.362302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdaf354-f8f2-491a-bf52-77d70e7031b2 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:15.368827 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea90c2-aedd-4c6d-83e1-a0a726831c09 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:15.370813 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea90c2-aedd-4c6d-83e1-a0a726831c09 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:15.376969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-dfdaf354-f8f2-491a-bf52-77d70e7031b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:26:15.376969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:15.376969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:15.376969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:15.376969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:15.376969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:15.376969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:15.376969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:15.376969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:15.376969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:15.376969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:15.376969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:15.376969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:26:15.376969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:26:15.376969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:26:15.377893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:26:15.377893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:15.377893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:15.377893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:15.377893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:15.377893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:15.377893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:15.377893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:15.377893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:26:15.377893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:15.377893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:15.377893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:15.377893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:15.377893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:15.377893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:15.377893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:15.377893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:15.378837 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:26:15.378837 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:26:15.378837 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:26:15.378837 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:26:15.378837 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:26:15.378837 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:26:15.378837 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:26:15.378837 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:15.382952 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 503/975] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:26:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:15.383797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdaf354-f8f2-491a-bf52-77d70e7031b2 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:15.390009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-d8ea90c2-aedd-4c6d-83e1-a0a726831c09 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 31 14:26:15.390009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:15.390009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:15.390009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:15.390009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:15.390009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:15.390009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:15.390009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:15.390009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:15.390009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:15.390009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:15.390009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:15.390009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:26:15.390009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:26:15.390009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:26:15.390009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:26:15.391053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:15.391053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:15.391053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:26:15.391053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:26:15.391053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:26:15.391053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:26:15.391053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:15.391053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:15.391053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:15.391053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:15.391053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:15.391053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:15.391053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:15.391053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:15.391053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:15.391053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:15.391053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:15.391053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:15.392037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:15.392037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:15.392037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:26:15.392037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:26:15.392037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:26:15.392037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:26:15.392037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:26:15.392037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:26:15.392037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 31 14:26:15.392037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:15.394658 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 473/976] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:26:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:15.399801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea90c2-aedd-4c6d-83e1-a0a726831c09 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:15.470533 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:15.502437 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdaf354-f8f2-491a-bf52-77d70e7031b2 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:15.531021 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea90c2-aedd-4c6d-83e1-a0a726831c09 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:15.541716 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4fb51-6d39-4d9e-bf06-b62e266cb3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:15.542400 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4fb51-6d39-4d9e-bf06-b62e266cb3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:15.542715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4fb51-6d39-4d9e-bf06-b62e266cb3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:15.543001 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4fb51-6d39-4d9e-bf06-b62e266cb3d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:15.543917 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4fb51-6d39-4d9e-bf06-b62e266cb3d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:15.544236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4fb51-6d39-4d9e-bf06-b62e266cb3d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:15.545797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4fb51-6d39-4d9e-bf06-b62e266cb3d3 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:15.558678 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 504/977] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:26:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:15.570669 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4fb51-6d39-4d9e-bf06-b62e266cb3d3 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:15.575947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a78e41-e057-4fed-8848-7a7592f4ff9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:15.576764 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a78e41-e057-4fed-8848-7a7592f4ff9b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:15.577234 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a78e41-e057-4fed-8848-7a7592f4ff9b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:15.577541 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a78e41-e057-4fed-8848-7a7592f4ff9b None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:15.578465 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a78e41-e057-4fed-8848-7a7592f4ff9b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:15.578924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a78e41-e057-4fed-8848-7a7592f4ff9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:15.580466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a78e41-e057-4fed-8848-7a7592f4ff9b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:15.620978 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 474/978] 10.222.0.67 () {56 vars in 1101 bytes} [Tue May 31 14:26:15 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:15.684678 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4fb51-6d39-4d9e-bf06-b62e266cb3d3 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:15.718492 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:15.724684 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a4536-025d-4e46-b39d-8f6eb0ebc1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:15.725368 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a4536-025d-4e46-b39d-8f6eb0ebc1ec None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:15.725652 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a4536-025d-4e46-b39d-8f6eb0ebc1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:15.725930 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a4536-025d-4e46-b39d-8f6eb0ebc1ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:15.726876 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a4536-025d-4e46-b39d-8f6eb0ebc1ec None admin] RBAC: Adding query filter params (role.id=8de91fffa27842109896bf18045db796, scope.project.id=f965a93a2f674598890bd8cafededb2c, user.id=0b7d100f2407432381d119c25814fff3) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:15.727252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a4536-025d-4e46-b39d-8f6eb0ebc1ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:15.729022 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a4536-025d-4e46-b39d-8f6eb0ebc1ec None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:15.742468 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 505/979] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:26:15 2022] GET /identity/v3/role_assignments?user.id=0b7d100f2407432381d119c25814fff3&scope.project.id=f965a93a2f674598890bd8cafededb2c&role.id=8de91fffa27842109896bf18045db796 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:16.479246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb7f77-cdf5-4fce-8027-e9de5b0d4798 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:16.479739 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb7f77-cdf5-4fce-8027-e9de5b0d4798 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:16.479739 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb7f77-cdf5-4fce-8027-e9de5b0d4798 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:16.481150 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 475/980] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:16.486365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ea3cf-2c2d-4f01-bafd-046479c632fb None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:16.486762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ea3cf-2c2d-4f01-bafd-046479c632fb None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:16.486964 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ea3cf-2c2d-4f01-bafd-046479c632fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:16.503627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-398ea3cf-2c2d-4f01-bafd-046479c632fb None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:16.504852 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-398ea3cf-2c2d-4f01-bafd-046479c632fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:16.573769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 506/981] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:16.598428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb7f77-cdf5-4fce-8027-e9de5b0d4798 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:16.626880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:16.693767 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f8fe1-a47f-4b37-ae5f-69d347b8f291 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:16.694050 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f8fe1-a47f-4b37-ae5f-69d347b8f291 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:16.694325 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f8fe1-a47f-4b37-ae5f-69d347b8f291 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:16.695747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 507/982] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:16.701688 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6651622-9c2b-40ca-8744-5fc98f344688 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:16.702103 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6651622-9c2b-40ca-8744-5fc98f344688 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:16.702484 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6651622-9c2b-40ca-8744-5fc98f344688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:16.703514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb7f77-cdf5-4fce-8027-e9de5b0d4798 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:16.718942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-d6651622-9c2b-40ca-8744-5fc98f344688 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:16.720136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d6651622-9c2b-40ca-8744-5fc98f344688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:16.744921 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a73d2-2e77-413f-98f1-62bded6cfac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:16.745518 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a73d2-2e77-413f-98f1-62bded6cfac4 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:16.745800 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a73d2-2e77-413f-98f1-62bded6cfac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:16.746074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a73d2-2e77-413f-98f1-62bded6cfac4 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:16.746948 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a73d2-2e77-413f-98f1-62bded6cfac4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:16.748429 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a73d2-2e77-413f-98f1-62bded6cfac4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:16.763031 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 476/983] 10.222.0.67 () {60 vars in 1141 bytes} [Tue May 31 14:26:16 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:26:16.797914 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 508/984] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:16 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 31 14:26:16.821048 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a73d2-2e77-413f-98f1-62bded6cfac4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:16.917976 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:16.936581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a73d2-2e77-413f-98f1-62bded6cfac4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:16.975936 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a4328-e169-400c-97b0-8e45b5737da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:16.976744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a4328-e169-400c-97b0-8e45b5737da7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:16.977069 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a4328-e169-400c-97b0-8e45b5737da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:16.977350 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a4328-e169-400c-97b0-8e45b5737da7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:16.978005 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85a4328-e169-400c-97b0-8e45b5737da7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:16.979440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85a4328-e169-400c-97b0-8e45b5737da7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:16.991437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f85a4328-e169-400c-97b0-8e45b5737da7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 31 14:26:16.991437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:16.991437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:26:16.991437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:26:16.991437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:26:16.991437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:26:16.991437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:26:16.991437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:16.991437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 31 14:26:16.991437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:16.991437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:26:16.991437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:16.991437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:16.991437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:16.991437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:16.991437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:16.991437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:16.992473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:16.992473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:16.992473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:16.992473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:16.992473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:16.992473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:16.992473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:26:16.992473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:26:16.992473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:26:16.992473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:26:16.992473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:16.992473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:16.992473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:16.992473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:16.992473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:16.992473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:16.993505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:16.993505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:16.993505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:16.993505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:16.993505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:16.993505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:16.993505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:16.993505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:16.993505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:16.993505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:16.993505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:26:16.993505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:16.993505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 31 14:26:16.993505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:16.993505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 477/985] 10.222.0.67 () {56 vars in 1126 bytes} [Tue May 31 14:26:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:16.998874 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6651622-9c2b-40ca-8744-5fc98f344688 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:17.053860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:17.103260 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6651622-9c2b-40ca-8744-5fc98f344688 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:17.146414 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce3f28-a947-4e4f-92d5-2e681adbb531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:17.147115 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce3f28-a947-4e4f-92d5-2e681adbb531 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:17.147406 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce3f28-a947-4e4f-92d5-2e681adbb531 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:17.147682 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce3f28-a947-4e4f-92d5-2e681adbb531 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:17.148392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce3f28-a947-4e4f-92d5-2e681adbb531 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:17.148700 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce3f28-a947-4e4f-92d5-2e681adbb531 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:17.150065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce3f28-a947-4e4f-92d5-2e681adbb531 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:17.159637 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 509/986] 10.222.0.67 () {56 vars in 1122 bytes} [Tue May 31 14:26:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:17.161158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a4328-e169-400c-97b0-8e45b5737da7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:17.262934 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a4328-e169-400c-97b0-8e45b5737da7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:17.298061 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ccc8f-557c-423a-a196-a1c51e6f3b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:17.298756 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ccc8f-557c-423a-a196-a1c51e6f3b4a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:17.299044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ccc8f-557c-423a-a196-a1c51e6f3b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:17.299324 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ccc8f-557c-423a-a196-a1c51e6f3b4a None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:17.300038 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ccc8f-557c-423a-a196-a1c51e6f3b4a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:17.300349 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ccc8f-557c-423a-a196-a1c51e6f3b4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:17.301726 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ccc8f-557c-423a-a196-a1c51e6f3b4a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:17.307557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 478/987] 10.222.0.67 () {56 vars in 1122 bytes} [Tue May 31 14:26:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:17.313120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce3f28-a947-4e4f-92d5-2e681adbb531 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:17.412197 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce3f28-a947-4e4f-92d5-2e681adbb531 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:17.428229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:17.447634 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08c2fd-bbb3-420a-9b41-cf2325106d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:17.448302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08c2fd-bbb3-420a-9b41-cf2325106d9c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:17.448585 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08c2fd-bbb3-420a-9b41-cf2325106d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:17.448860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08c2fd-bbb3-420a-9b41-cf2325106d9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:17.449664 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08c2fd-bbb3-420a-9b41-cf2325106d9c None admin] RBAC: Adding query filter params (interface=admin, service_id=c1e59e1d999c4b53870423df786441a3, region_id=RegionOne) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:17.449977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08c2fd-bbb3-420a-9b41-cf2325106d9c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:17.451631 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08c2fd-bbb3-420a-9b41-cf2325106d9c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:17.457203 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 510/988] 10.222.0.67 () {56 vars in 1249 bytes} [Tue May 31 14:26:17 2022] GET /identity/v3/endpoints?service_id=c1e59e1d999c4b53870423df786441a3&interface=admin®ion_id=RegionOne => generated 193 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:17.462637 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ccc8f-557c-423a-a196-a1c51e6f3b4a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:17.479198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:17.566376 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ccc8f-557c-423a-a196-a1c51e6f3b4a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:17.600828 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19164dc7-f704-4b62-952a-d36412a1959c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:17.601423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19164dc7-f704-4b62-952a-d36412a1959c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:17.601712 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19164dc7-f704-4b62-952a-d36412a1959c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:17.601990 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19164dc7-f704-4b62-952a-d36412a1959c None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:17.602692 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19164dc7-f704-4b62-952a-d36412a1959c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:17.604103 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19164dc7-f704-4b62-952a-d36412a1959c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:17.618742 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 479/989] 10.222.0.67 () {56 vars in 1087 bytes} [Tue May 31 14:26:17 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:26:18.062574 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86534398-9754-4a25-a1b2-c310faeb88b2 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:18.063140 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86534398-9754-4a25-a1b2-c310faeb88b2 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:18.063221 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86534398-9754-4a25-a1b2-c310faeb88b2 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:18.064717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 511/990] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:18.069739 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa71366-12d6-4a1e-99a5-77bf96551c91 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:18.070037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa71366-12d6-4a1e-99a5-77bf96551c91 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:18.070318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa71366-12d6-4a1e-99a5-77bf96551c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:18.087907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4aa71366-12d6-4a1e-99a5-77bf96551c91 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:18.089431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4aa71366-12d6-4a1e-99a5-77bf96551c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:18.158506 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 480/991] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:18.180294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86534398-9754-4a25-a1b2-c310faeb88b2 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:18.283665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86534398-9754-4a25-a1b2-c310faeb88b2 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:18.318319 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5dabc-7859-4a5a-bd7d-6f6af7aa6e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:18.319075 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5dabc-7859-4a5a-bd7d-6f6af7aa6e35 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:18.319361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5dabc-7859-4a5a-bd7d-6f6af7aa6e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:18.319637 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5dabc-7859-4a5a-bd7d-6f6af7aa6e35 None admin] PATH_INFO: `/v3/services/image` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:18.320294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e5dabc-7859-4a5a-bd7d-6f6af7aa6e35 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:18.321674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e5dabc-7859-4a5a-bd7d-6f6af7aa6e35 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:18.333714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-84e5dabc-7859-4a5a-bd7d-6f6af7aa6e35 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 31 14:26:18.333714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:18.333714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:26:18.333714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:26:18.333714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:26:18.333714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:26:18.333714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:26:18.333714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:18.333714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 31 14:26:18.333714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:18.333714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:26:18.333714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:18.333714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:18.333714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:18.333714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:18.333714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:18.333714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:18.334942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:18.334942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:18.334942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:18.334942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:18.334942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:18.334942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:18.334942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:26:18.334942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:26:18.334942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:26:18.334942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:26:18.334942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:18.334942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:18.334942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:18.334942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:18.334942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:18.334942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:18.335834 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:18.335834 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:26:18.335834 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:18.335834 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:18.335834 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:18.335834 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:18.335834 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:18.335834 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:18.335834 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:18.335834 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:18.335834 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:26:18.335834 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:18.335834 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 31 14:26:18.335834 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:18.338801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 512/992] 10.222.0.67 () {56 vars in 1105 bytes} [Tue May 31 14:26:18 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:18.341287 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa71366-12d6-4a1e-99a5-77bf96551c91 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:18.351426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:18.442373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa71366-12d6-4a1e-99a5-77bf96551c91 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:18.476418 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f00f7d-19c1-4100-83bb-375cfd9d38fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:18.477089 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f00f7d-19c1-4100-83bb-375cfd9d38fb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:18.477373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f00f7d-19c1-4100-83bb-375cfd9d38fb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:18.477650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f00f7d-19c1-4100-83bb-375cfd9d38fb None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:18.478914 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f00f7d-19c1-4100-83bb-375cfd9d38fb None admin] RBAC: Adding query filter params (name=image) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:18.479224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f00f7d-19c1-4100-83bb-375cfd9d38fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:18.480589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f00f7d-19c1-4100-83bb-375cfd9d38fb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:18.490023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 481/993] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:26:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:18.491322 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5dabc-7859-4a5a-bd7d-6f6af7aa6e35 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:18.541385 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:18.597525 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5dabc-7859-4a5a-bd7d-6f6af7aa6e35 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:18.631471 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df87c13-4810-4f04-9ca8-e6a2e3463fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:18.632133 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df87c13-4810-4f04-9ca8-e6a2e3463fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:18.632423 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df87c13-4810-4f04-9ca8-e6a2e3463fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:18.632701 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df87c13-4810-4f04-9ca8-e6a2e3463fd2 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:18.633411 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df87c13-4810-4f04-9ca8-e6a2e3463fd2 None admin] RBAC: Adding query filter params (type=image) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:18.633721 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df87c13-4810-4f04-9ca8-e6a2e3463fd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:18.635691 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df87c13-4810-4f04-9ca8-e6a2e3463fd2 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:18.646694 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 513/994] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:26:18 2022] GET /identity/v3/services?type=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:19.067338 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca7bf4-4adc-4bc7-bab7-b0d7af93d811 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:19.067856 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca7bf4-4adc-4bc7-bab7-b0d7af93d811 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:19.067935 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca7bf4-4adc-4bc7-bab7-b0d7af93d811 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:19.069415 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 482/995] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:19.074570 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c2ff6-b003-431a-9682-293bd82d9a9f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:19.074976 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c2ff6-b003-431a-9682-293bd82d9a9f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:19.075177 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c2ff6-b003-431a-9682-293bd82d9a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:19.092034 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-974c2ff6-b003-431a-9682-293bd82d9a9f None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:19.093233 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-974c2ff6-b003-431a-9682-293bd82d9a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:19.168001 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 514/996] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:19.191229 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca7bf4-4adc-4bc7-bab7-b0d7af93d811 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:19.264941 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:19.299205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca7bf4-4adc-4bc7-bab7-b0d7af93d811 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:19.334051 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475ef4c-2fe5-4c32-9e61-b75c6e2413c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:19.334811 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475ef4c-2fe5-4c32-9e61-b75c6e2413c5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:19.335100 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475ef4c-2fe5-4c32-9e61-b75c6e2413c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:19.335438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475ef4c-2fe5-4c32-9e61-b75c6e2413c5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:19.336098 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c475ef4c-2fe5-4c32-9e61-b75c6e2413c5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:19.338120 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c475ef4c-2fe5-4c32-9e61-b75c6e2413c5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:19.350028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-c475ef4c-2fe5-4c32-9e61-b75c6e2413c5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 31 14:26:19.350028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:19.350028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:26:19.350028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:26:19.350028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:26:19.350028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:26:19.350028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:26:19.350028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:19.350028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 31 14:26:19.350028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:19.350028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:26:19.350028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:19.350028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:19.350028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:19.350028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:19.350028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:19.350028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:19.351218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:19.351218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:19.351218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:19.351218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:19.351218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:19.351218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:19.351218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:26:19.351218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:26:19.351218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:26:19.351218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:26:19.351218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:19.351218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:19.351218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:19.351218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:19.351218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:19.351218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:19.352084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:19.352084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:19.352084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:19.352084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:19.352084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:19.352084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:19.352084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:19.352084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:19.352084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:19.352084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:19.352084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:26:19.352084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:19.352084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 31 14:26:19.352084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:19.354964 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 483/997] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:26:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:19.357844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c2ff6-b003-431a-9682-293bd82d9a9f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:19.397411 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:19.459695 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c2ff6-b003-431a-9682-293bd82d9a9f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:19.494669 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc6fc35-b691-48a7-9c3c-79dbd5c87cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:19.495414 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6fc35-b691-48a7-9c3c-79dbd5c87cba None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:19.495701 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6fc35-b691-48a7-9c3c-79dbd5c87cba None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:19.495980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6fc35-b691-48a7-9c3c-79dbd5c87cba None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:19.496744 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc6fc35-b691-48a7-9c3c-79dbd5c87cba None admin] RBAC: Adding query filter params (name=compute) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:19.497062 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc6fc35-b691-48a7-9c3c-79dbd5c87cba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:19.498444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc6fc35-b691-48a7-9c3c-79dbd5c87cba None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:19.508111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 515/998] 10.222.0.67 () {56 vars in 1112 bytes} [Tue May 31 14:26:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:19.509008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475ef4c-2fe5-4c32-9e61-b75c6e2413c5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:19.613850 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475ef4c-2fe5-4c32-9e61-b75c6e2413c5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:19.647082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a215a1-b6a8-48b7-868a-1d6c086ef8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:19.647747 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a215a1-b6a8-48b7-868a-1d6c086ef8a1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:19.648079 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a215a1-b6a8-48b7-868a-1d6c086ef8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:19.648363 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a215a1-b6a8-48b7-868a-1d6c086ef8a1 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:19.649068 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a215a1-b6a8-48b7-868a-1d6c086ef8a1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:19.649377 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a215a1-b6a8-48b7-868a-1d6c086ef8a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:19.651261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a215a1-b6a8-48b7-868a-1d6c086ef8a1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:19.653267 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:19.661894 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 484/999] 10.222.0.67 () {56 vars in 1112 bytes} [Tue May 31 14:26:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:19.663261 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc6fc35-b691-48a7-9c3c-79dbd5c87cba None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:19.764651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc6fc35-b691-48a7-9c3c-79dbd5c87cba None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:19.766231 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:19.800839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ef78c3-dfc7-405f-9a45-12cd2d3b62dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:19.801495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef78c3-dfc7-405f-9a45-12cd2d3b62dc None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:19.801776 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef78c3-dfc7-405f-9a45-12cd2d3b62dc None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:19.802048 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef78c3-dfc7-405f-9a45-12cd2d3b62dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:19.802873 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ef78c3-dfc7-405f-9a45-12cd2d3b62dc None admin] RBAC: Adding query filter params (interface=public, service_id=cadb7d337fe94d30885142bfd23612a0, region_id=RegionOne) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:19.803186 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ef78c3-dfc7-405f-9a45-12cd2d3b62dc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:19.804546 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ef78c3-dfc7-405f-9a45-12cd2d3b62dc None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:19.814204 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 516/1000] 10.222.0.67 () {56 vars in 1251 bytes} [Tue May 31 14:26:19 2022] GET /identity/v3/endpoints?service_id=cadb7d337fe94d30885142bfd23612a0&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:19.815409 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a215a1-b6a8-48b7-868a-1d6c086ef8a1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:19.880863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69803d5-f101-40ea-94ad-c2ff719b05ef None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:19.881162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69803d5-f101-40ea-94ad-c2ff719b05ef None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:19.881333 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69803d5-f101-40ea-94ad-c2ff719b05ef None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:19.882914 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 517/1001] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:19.888228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab412fa-6a5c-486b-a87d-61f5fa286120 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:19.888517 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab412fa-6a5c-486b-a87d-61f5fa286120 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:19.888798 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab412fa-6a5c-486b-a87d-61f5fa286120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:19.905881 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-1ab412fa-6a5c-486b-a87d-61f5fa286120 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:19.907120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1ab412fa-6a5c-486b-a87d-61f5fa286120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:19.920233 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a215a1-b6a8-48b7-868a-1d6c086ef8a1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:19.957582 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504148a1-a88f-4123-99e4-b9c305f57fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:19.958252 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504148a1-a88f-4123-99e4-b9c305f57fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:19.958539 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504148a1-a88f-4123-99e4-b9c305f57fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:19.959099 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504148a1-a88f-4123-99e4-b9c305f57fd7 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:19.959524 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504148a1-a88f-4123-99e4-b9c305f57fd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:19.960897 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504148a1-a88f-4123-99e4-b9c305f57fd7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:19.975156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 485/1002] 10.222.0.67 () {56 vars in 1087 bytes} [Tue May 31 14:26:19 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:26:19.977152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 518/1003] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 31 14:26:20.000407 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504148a1-a88f-4123-99e4-b9c305f57fd7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:20.069702 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:20.113217 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504148a1-a88f-4123-99e4-b9c305f57fd7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:20.155222 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c097221-31b1-409e-8bc0-8e2fe58ef842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:20.155944 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c097221-31b1-409e-8bc0-8e2fe58ef842 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:20.156897 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c097221-31b1-409e-8bc0-8e2fe58ef842 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:20.156897 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c097221-31b1-409e-8bc0-8e2fe58ef842 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:20.157152 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c097221-31b1-409e-8bc0-8e2fe58ef842 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:20.158616 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c097221-31b1-409e-8bc0-8e2fe58ef842 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:20.172139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-2c097221-31b1-409e-8bc0-8e2fe58ef842 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 31 14:26:20.172139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:20.172139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:26:20.172139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:26:20.172139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:26:20.172139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:26:20.172139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:26:20.172139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:20.172139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 31 14:26:20.172139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:20.172139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:26:20.172139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:20.172139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:20.172139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:20.172139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:20.172139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:20.172139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:20.173151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:20.173151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:20.173151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:20.173151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:20.173151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:20.173151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:20.173151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:26:20.173151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:26:20.173151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:26:20.173151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:26:20.173151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:20.173151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:20.173151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:20.173151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:20.173151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:20.173151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:20.174161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:20.174161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:20.174161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:20.174161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:20.174161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:20.174161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:20.174161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:20.174161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:20.174161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:20.174161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:20.174161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:26:20.174161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:20.174161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 31 14:26:20.174161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:20.174161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 486/1004] 10.222.0.67 () {56 vars in 1126 bytes} [Tue May 31 14:26:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:20.179430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab412fa-6a5c-486b-a87d-61f5fa286120 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:20.277279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:20.283833 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab412fa-6a5c-486b-a87d-61f5fa286120 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:20.317737 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0890cf9-c356-4944-9a36-198ab4396ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:20.318401 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0890cf9-c356-4944-9a36-198ab4396ada None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:20.318708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0890cf9-c356-4944-9a36-198ab4396ada None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:20.318991 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0890cf9-c356-4944-9a36-198ab4396ada None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:20.319699 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0890cf9-c356-4944-9a36-198ab4396ada None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:20.320008 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0890cf9-c356-4944-9a36-198ab4396ada None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:20.321373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0890cf9-c356-4944-9a36-198ab4396ada None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:20.327288 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 519/1005] 10.222.0.67 () {56 vars in 1122 bytes} [Tue May 31 14:26:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:20.332714 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c097221-31b1-409e-8bc0-8e2fe58ef842 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:20.436985 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c097221-31b1-409e-8bc0-8e2fe58ef842 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:20.463573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:20.472868 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4489b4a-9b3e-4737-92d5-926625705e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:20.473551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4489b4a-9b3e-4737-92d5-926625705e73 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:20.473833 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4489b4a-9b3e-4737-92d5-926625705e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:20.474108 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4489b4a-9b3e-4737-92d5-926625705e73 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:20.474856 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4489b4a-9b3e-4737-92d5-926625705e73 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:20.475159 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4489b4a-9b3e-4737-92d5-926625705e73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:20.476522 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4489b4a-9b3e-4737-92d5-926625705e73 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:20.486810 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 487/1006] 10.222.0.67 () {56 vars in 1122 bytes} [Tue May 31 14:26:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:20.488323 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0890cf9-c356-4944-9a36-198ab4396ada None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:20.599656 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0890cf9-c356-4944-9a36-198ab4396ada None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:20.634943 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba11f49-2cb2-4121-8f5a-9f6345728b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:20.635607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba11f49-2cb2-4121-8f5a-9f6345728b1b None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:20.635892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba11f49-2cb2-4121-8f5a-9f6345728b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:20.636171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba11f49-2cb2-4121-8f5a-9f6345728b1b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:20.636908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba11f49-2cb2-4121-8f5a-9f6345728b1b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:20.638303 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba11f49-2cb2-4121-8f5a-9f6345728b1b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:20.662682 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 520/1007] 10.222.0.67 () {60 vars in 1144 bytes} [Tue May 31 14:26:20 2022] POST /identity/v3/endpoints => generated 325 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:26:20.896320 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039ce24-ceee-42ca-88d3-f7e3229811ed None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:20.896605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039ce24-ceee-42ca-88d3-f7e3229811ed None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:20.896882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039ce24-ceee-42ca-88d3-f7e3229811ed None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:20.898256 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 488/1008] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:20.904208 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798323b8-c1a1-466f-a575-d8263f4c21a0 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:20.904506 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798323b8-c1a1-466f-a575-d8263f4c21a0 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:20.904792 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798323b8-c1a1-466f-a575-d8263f4c21a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:20.921271 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-798323b8-c1a1-466f-a575-d8263f4c21a0 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:20.922495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-798323b8-c1a1-466f-a575-d8263f4c21a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:20.937848 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:21.000041 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 521/1009] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:20 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:21.023375 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039ce24-ceee-42ca-88d3-f7e3229811ed None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:21.133205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039ce24-ceee-42ca-88d3-f7e3229811ed None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:21.170474 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc0379-f3d5-4a16-b8ad-36868fa9df14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:21.171109 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc0379-f3d5-4a16-b8ad-36868fa9df14 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:21.171395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc0379-f3d5-4a16-b8ad-36868fa9df14 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:21.171672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc0379-f3d5-4a16-b8ad-36868fa9df14 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:21.172404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc0379-f3d5-4a16-b8ad-36868fa9df14 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:21.173794 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc0379-f3d5-4a16-b8ad-36868fa9df14 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:21.192332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 489/1010] 10.222.0.67 () {60 vars in 1141 bytes} [Tue May 31 14:26:21 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:26:22.280198 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47d01b-cfbb-42f2-be0e-3b56385d88f5 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:22.280756 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47d01b-cfbb-42f2-be0e-3b56385d88f5 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:22.280756 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47d01b-cfbb-42f2-be0e-3b56385d88f5 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:22.282188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 522/1011] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:22.287437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b46fbc-c138-4b13-9770-96ebbd1a0399 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:22.287833 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b46fbc-c138-4b13-9770-96ebbd1a0399 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:22.287994 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b46fbc-c138-4b13-9770-96ebbd1a0399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:22.305198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b8b46fbc-c138-4b13-9770-96ebbd1a0399 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:22.307029 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b8b46fbc-c138-4b13-9770-96ebbd1a0399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:22.359929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:22.390437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 490/1012] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:22 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:22.413760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf47d01b-cfbb-42f2-be0e-3b56385d88f5 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:22.517682 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf47d01b-cfbb-42f2-be0e-3b56385d88f5 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:22.551823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19044602-6c8e-41ad-a225-f415c311a428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:22.552617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19044602-6c8e-41ad-a225-f415c311a428 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:22.552900 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19044602-6c8e-41ad-a225-f415c311a428 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:22.553172 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19044602-6c8e-41ad-a225-f415c311a428 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:22.553823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19044602-6c8e-41ad-a225-f415c311a428 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:22.555326 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19044602-6c8e-41ad-a225-f415c311a428 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:22.567229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-19044602-6c8e-41ad-a225-f415c311a428 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 31 14:26:22.567229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:22.567229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:26:22.567229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:26:22.567229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:26:22.567229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:26:22.567229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:26:22.567229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:22.567229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 31 14:26:22.567229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:22.567229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:26:22.567229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:22.567229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:22.567229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:22.567229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:22.567229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:22.567229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:22.568318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:22.568318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:22.568318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:22.568318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:22.568318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:22.568318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:22.568318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:26:22.568318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:26:22.568318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:26:22.568318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:26:22.568318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:22.568318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:22.568318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:22.568318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:22.568318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:22.568318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:22.569184 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:22.569184 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:26:22.569184 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:22.569184 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:22.569184 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:22.569184 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:22.569184 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:22.569184 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:22.569184 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:22.569184 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:22.569184 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:26:22.569184 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:22.569184 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 31 14:26:22.569184 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:22.571023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 523/1013] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:26:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:22.574904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b46fbc-c138-4b13-9770-96ebbd1a0399 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:22.678258 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b46fbc-c138-4b13-9770-96ebbd1a0399 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:22.714630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63475308-a0e6-4080-974d-8befa7c1d2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:22.715321 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63475308-a0e6-4080-974d-8befa7c1d2f8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:22.715606 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63475308-a0e6-4080-974d-8befa7c1d2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:22.715883 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63475308-a0e6-4080-974d-8befa7c1d2f8 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:22.716591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63475308-a0e6-4080-974d-8befa7c1d2f8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:22.716902 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63475308-a0e6-4080-974d-8befa7c1d2f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:22.718262 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63475308-a0e6-4080-974d-8befa7c1d2f8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:22.729948 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 491/1014] 10.222.0.67 () {56 vars in 1112 bytes} [Tue May 31 14:26:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:22.730114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19044602-6c8e-41ad-a225-f415c311a428 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:22.742785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:22.832772 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19044602-6c8e-41ad-a225-f415c311a428 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:22.865906 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d96cde-520e-46ec-b5bb-eaa796ddc235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:22.866570 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d96cde-520e-46ec-b5bb-eaa796ddc235 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:22.866880 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d96cde-520e-46ec-b5bb-eaa796ddc235 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:22.867157 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d96cde-520e-46ec-b5bb-eaa796ddc235 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:22.867859 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d96cde-520e-46ec-b5bb-eaa796ddc235 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:22.868164 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d96cde-520e-46ec-b5bb-eaa796ddc235 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:22.869556 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d96cde-520e-46ec-b5bb-eaa796ddc235 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:22.879145 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 524/1015] 10.222.0.67 () {56 vars in 1112 bytes} [Tue May 31 14:26:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:22.880365 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63475308-a0e6-4080-974d-8befa7c1d2f8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:22.908634 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4a742-039d-4c83-89e0-d1171d5e2d9d None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:22.908922 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4a742-039d-4c83-89e0-d1171d5e2d9d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:22.909200 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4a742-039d-4c83-89e0-d1171d5e2d9d None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:22.910573 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 525/1016] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 31 14:26:22.915525 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2340d-6db8-41db-8cb1-fdabc66f07a6 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:22.915819 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2340d-6db8-41db-8cb1-fdabc66f07a6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:22.916093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2340d-6db8-41db-8cb1-fdabc66f07a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:22.931464 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:22.932657 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-92a2340d-6db8-41db-8cb1-fdabc66f07a6 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:22.933850 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-92a2340d-6db8-41db-8cb1-fdabc66f07a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:22.996000 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63475308-a0e6-4080-974d-8befa7c1d2f8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:23.004320 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 526/1017] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:22 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 31 14:26:23.028230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2340d-6db8-41db-8cb1-fdabc66f07a6 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:23.031456 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc84ff2-9709-4308-a615-053f81a10787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:23.032119 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc84ff2-9709-4308-a615-053f81a10787 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:23.032404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc84ff2-9709-4308-a615-053f81a10787 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:23.032679 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc84ff2-9709-4308-a615-053f81a10787 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:23.032914 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:23.033415 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc84ff2-9709-4308-a615-053f81a10787 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:23.034848 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc84ff2-9709-4308-a615-053f81a10787 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:23.054086 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 492/1018] 10.222.0.67 () {60 vars in 1144 bytes} [Tue May 31 14:26:22 2022] POST /identity/v3/endpoints => generated 334 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:26:23.157715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2340d-6db8-41db-8cb1-fdabc66f07a6 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:23.202533 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cdbc9f-d18f-418f-8219-805f3bd5db1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:23.203297 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdbc9f-d18f-418f-8219-805f3bd5db1d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:23.203581 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdbc9f-d18f-418f-8219-805f3bd5db1d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:23.203855 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdbc9f-d18f-418f-8219-805f3bd5db1d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:23.205118 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cdbc9f-d18f-418f-8219-805f3bd5db1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:23.206597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cdbc9f-d18f-418f-8219-805f3bd5db1d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:23.213018 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 527/1019] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:26:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:23.214208 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc84ff2-9709-4308-a615-053f81a10787 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:23.320334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:23.325750 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc84ff2-9709-4308-a615-053f81a10787 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:23.361589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37153450-c23b-4687-a218-0f16f6f9b086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:23.362263 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37153450-c23b-4687-a218-0f16f6f9b086 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:23.362549 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37153450-c23b-4687-a218-0f16f6f9b086 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:23.362875 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37153450-c23b-4687-a218-0f16f6f9b086 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:23.363596 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37153450-c23b-4687-a218-0f16f6f9b086 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:23.365063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37153450-c23b-4687-a218-0f16f6f9b086 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:23.393404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 493/1020] 10.222.0.67 () {60 vars in 1141 bytes} [Tue May 31 14:26:23 2022] POST /identity/v3/projects => generated 311 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:26:23.472926 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f27256-ab50-4b96-8c89-5f9179c7e5df None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:23.473445 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f27256-ab50-4b96-8c89-5f9179c7e5df None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:23.473445 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f27256-ab50-4b96-8c89-5f9179c7e5df None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:23.475040 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 528/1021] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:23.481371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdf61d-e590-4569-a4be-ae40db1392e5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:23.481682 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdf61d-e590-4569-a4be-ae40db1392e5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:23.481948 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdf61d-e590-4569-a4be-ae40db1392e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:23.500166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b2fdf61d-e590-4569-a4be-ae40db1392e5 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:23.501413 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b2fdf61d-e590-4569-a4be-ae40db1392e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:23.576844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 494/1022] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:23.600339 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f27256-ab50-4b96-8c89-5f9179c7e5df None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:23.675005 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:23.711410 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f27256-ab50-4b96-8c89-5f9179c7e5df None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:23.745878 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3682de1-1da0-4d9d-816b-8107e5e91fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:23.746592 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3682de1-1da0-4d9d-816b-8107e5e91fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:23.746907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3682de1-1da0-4d9d-816b-8107e5e91fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:23.747184 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3682de1-1da0-4d9d-816b-8107e5e91fa6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:23.747841 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3682de1-1da0-4d9d-816b-8107e5e91fa6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:23.749225 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3682de1-1da0-4d9d-816b-8107e5e91fa6 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:23.761361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-a3682de1-1da0-4d9d-816b-8107e5e91fa6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 31 14:26:23.761361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:23.761361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:26:23.761361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:26:23.761361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:26:23.761361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:26:23.761361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:26:23.761361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:23.761361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 31 14:26:23.761361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:23.761361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:26:23.761361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:23.761361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:23.761361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:23.761361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:23.761361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:23.761361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:23.762483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:23.762483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:23.762483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:23.762483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:23.762483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:23.762483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:23.762483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:26:23.762483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:26:23.762483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:26:23.762483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:26:23.762483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:23.762483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:23.762483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:23.762483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:23.762483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:23.762483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:23.763530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:23.763530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:26:23.763530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:23.763530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:23.763530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:23.763530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:23.763530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:23.763530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:23.763530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:23.763530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:23.763530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:26:23.763530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:23.763530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 31 14:26:23.763530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:23.763530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 529/1023] 10.222.0.67 () {56 vars in 1105 bytes} [Tue May 31 14:26:23 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:23.768530 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fdf61d-e590-4569-a4be-ae40db1392e5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:23.851986 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:23.873657 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fdf61d-e590-4569-a4be-ae40db1392e5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:23.908351 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98563d36-2c93-4804-9cf1-689d563bad2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:23.909029 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98563d36-2c93-4804-9cf1-689d563bad2d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:23.909315 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98563d36-2c93-4804-9cf1-689d563bad2d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:23.909596 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98563d36-2c93-4804-9cf1-689d563bad2d None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:23.910304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98563d36-2c93-4804-9cf1-689d563bad2d None admin] RBAC: Adding query filter params (name=image) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:23.910612 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98563d36-2c93-4804-9cf1-689d563bad2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:23.912010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98563d36-2c93-4804-9cf1-689d563bad2d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:23.921769 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 495/1024] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:26:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:23.922979 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3682de1-1da0-4d9d-816b-8107e5e91fa6 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:24.031997 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3682de1-1da0-4d9d-816b-8107e5e91fa6 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:24.061987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:24.068171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41159500-94d8-4db9-a853-c820875a649b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:24.068840 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41159500-94d8-4db9-a853-c820875a649b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:24.069127 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41159500-94d8-4db9-a853-c820875a649b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:24.069405 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41159500-94d8-4db9-a853-c820875a649b None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:24.070121 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41159500-94d8-4db9-a853-c820875a649b None admin] RBAC: Adding query filter params (type=image) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:24.070431 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41159500-94d8-4db9-a853-c820875a649b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:24.072384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41159500-94d8-4db9-a853-c820875a649b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:24.077859 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 530/1025] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:26:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:24.083547 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98563d36-2c93-4804-9cf1-689d563bad2d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:24.190059 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98563d36-2c93-4804-9cf1-689d563bad2d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:24.224650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81baa6fa-8afd-4e81-a6ab-8e895d8ea58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:24.225315 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81baa6fa-8afd-4e81-a6ab-8e895d8ea58f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:24.225602 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81baa6fa-8afd-4e81-a6ab-8e895d8ea58f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:24.225881 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81baa6fa-8afd-4e81-a6ab-8e895d8ea58f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:24.226689 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81baa6fa-8afd-4e81-a6ab-8e895d8ea58f None admin] RBAC: Adding query filter params (interface=public, service_id=a7e0b160698c4fa4b09b74a08ec53f63, region_id=RegionOne) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:24.227010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81baa6fa-8afd-4e81-a6ab-8e895d8ea58f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:24.228910 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81baa6fa-8afd-4e81-a6ab-8e895d8ea58f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:24.238731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 496/1026] 10.222.0.67 () {56 vars in 1251 bytes} [Tue May 31 14:26:24 2022] GET /identity/v3/endpoints?service_id=a7e0b160698c4fa4b09b74a08ec53f63&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:24.239776 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41159500-94d8-4db9-a853-c820875a649b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:24.341496 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41159500-94d8-4db9-a853-c820875a649b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:24.379217 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db11387-fd8b-47d8-8dda-615b98b6e5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:24.379883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db11387-fd8b-47d8-8dda-615b98b6e5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:24.380164 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db11387-fd8b-47d8-8dda-615b98b6e5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:24.380436 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db11387-fd8b-47d8-8dda-615b98b6e5a7 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:24.381117 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db11387-fd8b-47d8-8dda-615b98b6e5a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:24.382489 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db11387-fd8b-47d8-8dda-615b98b6e5a7 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:24.389176 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 531/1027] 10.222.0.67 () {56 vars in 1087 bytes} [Tue May 31 14:26:24 2022] GET /identity/v3/services => generated 1957 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:26:25.476410 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf189d21-6aff-4389-abc5-a5369b7b6a79 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:25.476896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf189d21-6aff-4389-abc5-a5369b7b6a79 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:25.476896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf189d21-6aff-4389-abc5-a5369b7b6a79 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:25.478300 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 497/1028] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:25.483618 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114834f-363c-493b-bc47-f94b2304fa63 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:25.484014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114834f-363c-493b-bc47-f94b2304fa63 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:25.484181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114834f-363c-493b-bc47-f94b2304fa63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:25.502735 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-4114834f-363c-493b-bc47-f94b2304fa63 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:25.503935 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4114834f-363c-493b-bc47-f94b2304fa63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:25.561325 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:25.574409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 532/1029] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:25.597907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf189d21-6aff-4389-abc5-a5369b7b6a79 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:25.609233 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:25.702288 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf189d21-6aff-4389-abc5-a5369b7b6a79 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:25.735562 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec17673-3215-44c9-afdd-afad60688de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:25.736303 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec17673-3215-44c9-afdd-afad60688de2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:25.736585 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec17673-3215-44c9-afdd-afad60688de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:25.736859 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec17673-3215-44c9-afdd-afad60688de2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:25.748110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec17673-3215-44c9-afdd-afad60688de2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:26:25.756064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-2ec17673-3215-44c9-afdd-afad60688de2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 31 14:26:25.756064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:25.756064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:25.756064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:25.756064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:25.756064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:25.756064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:25.756064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:25.756064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:25.756064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:25.756064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:25.756064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:25.756064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:26:25.756064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:26:25.756064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:26:25.756064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:26:25.757196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:26:25.757196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:26:25.757196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:26:25.757196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:26:25.757196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:25.757196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:25.757196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:25.757196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:25.757196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:25.757196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:25.757196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:25.757196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:25.757196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:25.757196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:25.757196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:25.757196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:25.758274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:25.758274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:25.758274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:25.758274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:25.758274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:26:25.758274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:26:25.758274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:26:25.758274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:26:25.758274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:26:25.758274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:26:25.758274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 31 14:26:25.758274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:25.760456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2985d-eba0-480a-b02b-e8ce956cc484 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:25.760755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2985d-eba0-480a-b02b-e8ce956cc484 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:25.761044 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2985d-eba0-480a-b02b-e8ce956cc484 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:25.762008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 498/1030] 10.222.0.67 () {56 vars in 1120 bytes} [Tue May 31 14:26:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:25.762773 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 533/1031] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:25.766438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec17673-3215-44c9-afdd-afad60688de2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:25.768041 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ead716-4e8a-4fd5-abb3-e60b64f1a603 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:25.768337 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ead716-4e8a-4fd5-abb3-e60b64f1a603 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:25.768624 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ead716-4e8a-4fd5-abb3-e60b64f1a603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:25.785783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-17ead716-4e8a-4fd5-abb3-e60b64f1a603 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:25.787009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-17ead716-4e8a-4fd5-abb3-e60b64f1a603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:25.829299 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:25.858549 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 534/1032] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 31 14:26:25.867452 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec17673-3215-44c9-afdd-afad60688de2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:25.883201 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ead716-4e8a-4fd5-abb3-e60b64f1a603 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:25.906078 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b045707-eb23-4463-b649-dbf192ecf1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:25.906933 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b045707-eb23-4463-b649-dbf192ecf1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:25.907220 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b045707-eb23-4463-b649-dbf192ecf1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:25.907816 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b045707-eb23-4463-b649-dbf192ecf1d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:25.908851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b045707-eb23-4463-b649-dbf192ecf1d4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:25.909163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b045707-eb23-4463-b649-dbf192ecf1d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:25.911302 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b045707-eb23-4463-b649-dbf192ecf1d4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:25.926596 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 499/1033] 10.222.0.67 () {56 vars in 1115 bytes} [Tue May 31 14:26:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:25.927906 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b045707-eb23-4463-b649-dbf192ecf1d4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:25.986808 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ead716-4e8a-4fd5-abb3-e60b64f1a603 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:26.007705 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:26.022941 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877369c-cee3-4159-9324-9eb560d32fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:26.023740 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877369c-cee3-4159-9324-9eb560d32fca None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:26.024027 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877369c-cee3-4159-9324-9eb560d32fca None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:26.024304 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877369c-cee3-4159-9324-9eb560d32fca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:26.025518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877369c-cee3-4159-9324-9eb560d32fca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:26.026979 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877369c-cee3-4159-9324-9eb560d32fca None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:26.028615 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 535/1034] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:26:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:26.034250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877369c-cee3-4159-9324-9eb560d32fca None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:26.046473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b045707-eb23-4463-b649-dbf192ecf1d4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:26.075755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:26.083630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b9307-4ef7-442e-8909-d35d11540b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:26.084439 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b9307-4ef7-442e-8909-d35d11540b33 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:26.084726 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b9307-4ef7-442e-8909-d35d11540b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:26.085003 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b9307-4ef7-442e-8909-d35d11540b33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:26.096419 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b9307-4ef7-442e-8909-d35d11540b33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:26.097871 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b9307-4ef7-442e-8909-d35d11540b33 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:26.115544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-bf6b9307-4ef7-442e-8909-d35d11540b33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:26.115544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:26.115544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:26.115544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:26.115544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:26.115544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:26.115544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:26.115544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:26.115544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:26.115544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:26.115544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:26.115544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:26.115544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:26:26.115544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:26:26.115544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:26:26.116490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:26:26.116490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:26.116490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:26.116490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:26.116490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:26.116490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:26.116490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:26.116490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:26.116490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:26.116490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:26.116490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:26.116490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:26.116490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:26.116490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:26.116490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:26.116490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:26.116490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:26.124211 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:26:26.124211 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:26:26.124211 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:26.124211 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:26.124211 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 500/1035] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:26:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:26.124211 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b9307-4ef7-442e-8909-d35d11540b33 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:26.155536 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877369c-cee3-4159-9324-9eb560d32fca None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:26.191311 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b1a73-6419-4f7a-a492-39fb557ef90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:26.192334 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b1a73-6419-4f7a-a492-39fb557ef90b None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:26.192723 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b1a73-6419-4f7a-a492-39fb557ef90b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:26.193076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b1a73-6419-4f7a-a492-39fb557ef90b None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:26.193790 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b1a73-6419-4f7a-a492-39fb557ef90b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:26.195371 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b1a73-6419-4f7a-a492-39fb557ef90b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:26.226691 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 536/1036] 10.222.0.67 () {60 vars in 1132 bytes} [Tue May 31 14:26:26 2022] POST /identity/v3/users => generated 285 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:26:26.245591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b9307-4ef7-442e-8909-d35d11540b33 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:26.283280 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed003e53-d0c9-44c3-b102-4c958b7ae618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:26.283962 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed003e53-d0c9-44c3-b102-4c958b7ae618 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:26.284251 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed003e53-d0c9-44c3-b102-4c958b7ae618 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:26.284536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed003e53-d0c9-44c3-b102-4c958b7ae618 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:26.285279 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed003e53-d0c9-44c3-b102-4c958b7ae618 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:26.285592 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed003e53-d0c9-44c3-b102-4c958b7ae618 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:26.287046 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed003e53-d0c9-44c3-b102-4c958b7ae618 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:26.301412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 501/1037] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:26.307972 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b1a73-6419-4f7a-a492-39fb557ef90b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:26.361857 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:26.423287 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b1a73-6419-4f7a-a492-39fb557ef90b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:26.461103 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1a292-5146-46b6-ad1b-04e82d9071fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:26.462829 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1a292-5146-46b6-ad1b-04e82d9071fa None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:26.463119 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1a292-5146-46b6-ad1b-04e82d9071fa None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:26.463396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1a292-5146-46b6-ad1b-04e82d9071fa None admin] PATH_INFO: `/v3/users/nova` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:26.471562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a1a292-5146-46b6-ad1b-04e82d9071fa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:26.473162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a1a292-5146-46b6-ad1b-04e82d9071fa None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:26.488135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-97a1a292-5146-46b6-ad1b-04e82d9071fa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 31 14:26:26.488135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:26.488135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:26.488135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:26.488135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:26.488135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:26.488135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:26.488135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:26.488135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:26.488135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:26.488135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:26.488135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:26.488135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:26:26.488135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:26:26.488135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:26:26.488135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:26:26.489518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:26.489518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:26.489518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:26:26.489518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:26:26.489518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:26:26.489518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:26:26.489518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:26.489518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:26.489518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:26.489518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:26.489518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:26.489518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:26:26.489518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:26.489518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:26.489518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:26.489518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:26.489518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:26.489518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:26.490491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:26.490491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:26.490491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:26:26.490491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:26:26.490491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:26:26.490491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:26:26.490491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:26:26.490491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:26:26.490491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 31 14:26:26.490491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:26.490491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 537/1038] 10.222.0.67 () {56 vars in 1093 bytes} [Tue May 31 14:26:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:26.495407 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed003e53-d0c9-44c3-b102-4c958b7ae618 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:26.541453 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:26.600142 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed003e53-d0c9-44c3-b102-4c958b7ae618 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:26.633385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d9cb4-2fe7-4ac6-97fb-e0adca7d60dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:26.634568 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d9cb4-2fe7-4ac6-97fb-e0adca7d60dc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:26.634899 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d9cb4-2fe7-4ac6-97fb-e0adca7d60dc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:26.635177 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d9cb4-2fe7-4ac6-97fb-e0adca7d60dc None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:26.636012 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d9cb4-2fe7-4ac6-97fb-e0adca7d60dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:26.636325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d9cb4-2fe7-4ac6-97fb-e0adca7d60dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:26.637754 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d9cb4-2fe7-4ac6-97fb-e0adca7d60dc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:26.654088 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11ca0f-1b8f-4154-b0e7-b72ad5674e93 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:26.654383 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11ca0f-1b8f-4154-b0e7-b72ad5674e93 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:26.654672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11ca0f-1b8f-4154-b0e7-b72ad5674e93 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:26.656211 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 538/1039] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:26.661207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 502/1040] 10.222.0.67 () {56 vars in 1133 bytes} [Tue May 31 14:26:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:26.661333 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb81906-4ca7-4b24-be45-6a3536bfb596 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:26.661589 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb81906-4ca7-4b24-be45-6a3536bfb596 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:26.661868 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb81906-4ca7-4b24-be45-6a3536bfb596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:26.662147 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d9cb4-2fe7-4ac6-97fb-e0adca7d60dc None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:26.679540 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-acb81906-4ca7-4b24-be45-6a3536bfb596 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:26.680738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-acb81906-4ca7-4b24-be45-6a3536bfb596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:26.754502 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 539/1041] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:26.768604 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:26.772250 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d9cb4-2fe7-4ac6-97fb-e0adca7d60dc None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:26.779066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb81906-4ca7-4b24-be45-6a3536bfb596 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:26.812254 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27d001-ed2f-4c18-ab0d-7b33874948e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:26.812446 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:26.812834 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27d001-ed2f-4c18-ab0d-7b33874948e1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:26.813123 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27d001-ed2f-4c18-ab0d-7b33874948e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:26.813400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27d001-ed2f-4c18-ab0d-7b33874948e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:26.884900 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb81906-4ca7-4b24-be45-6a3536bfb596 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:26.890037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce27d001-ed2f-4c18-ab0d-7b33874948e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:26.891533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce27d001-ed2f-4c18-ab0d-7b33874948e1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:26.921658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d3576-063b-433c-8d66-b2a4da11ecb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:26.922450 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d3576-063b-433c-8d66-b2a4da11ecb4 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:26.922767 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d3576-063b-433c-8d66-b2a4da11ecb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:26.923048 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d3576-063b-433c-8d66-b2a4da11ecb4 None admin] PATH_INFO: `/v3/services/image` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:26.923831 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d3576-063b-433c-8d66-b2a4da11ecb4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:26.925202 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d3576-063b-433c-8d66-b2a4da11ecb4 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:26.938054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-ee1d3576-063b-433c-8d66-b2a4da11ecb4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 31 14:26:26.938054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:26.938054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:26:26.938054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:26:26.938054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:26:26.938054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:26:26.938054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:26:26.938054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:26.938054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 31 14:26:26.938054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:26.938054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:26:26.938054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:26.938054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:26.938054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:26.938054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:26.938054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:26.938054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:26.939120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:26.939120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:26.939120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:26.939120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:26.939120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:26.939120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:26.939120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:26:26.939120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:26:26.939120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:26:26.939120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:26:26.939120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:26.939120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:26.939120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:26.939120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:26.939120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:26.939120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:26.939999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:26.939999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:26:26.939999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:26.939999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:26.939999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:26.939999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:26.939999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:26.939999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:26.939999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:26.939999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:26.939999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:26:26.939999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:26:26.939999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 31 14:26:26.939999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:26.939999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 540/1042] 10.222.0.67 () {56 vars in 1105 bytes} [Tue May 31 14:26:26 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:26:26.945164 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d3576-063b-433c-8d66-b2a4da11ecb4 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:26.995974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:27.000818 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 503/1043] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:26:26 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 31 14:26:27.006029 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27d001-ed2f-4c18-ab0d-7b33874948e1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:27.057475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:27.062384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d3576-063b-433c-8d66-b2a4da11ecb4 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:27.106812 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da7c46-5479-4c1f-91aa-fff7f334af9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:27.107516 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da7c46-5479-4c1f-91aa-fff7f334af9c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:27.107776 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da7c46-5479-4c1f-91aa-fff7f334af9c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:27.108052 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da7c46-5479-4c1f-91aa-fff7f334af9c None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:27.108764 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da7c46-5479-4c1f-91aa-fff7f334af9c None admin] RBAC: Adding query filter params (name=image) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:27.109075 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da7c46-5479-4c1f-91aa-fff7f334af9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:27.110462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da7c46-5479-4c1f-91aa-fff7f334af9c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:27.115678 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 541/1044] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:26:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:27.121867 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da7c46-5479-4c1f-91aa-fff7f334af9c None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:27.122926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27d001-ed2f-4c18-ab0d-7b33874948e1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:27.160469 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdcfbaa-91a4-4b26-bbf5-856ef95e31b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:27.161265 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdcfbaa-91a4-4b26-bbf5-856ef95e31b9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:27.161550 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdcfbaa-91a4-4b26-bbf5-856ef95e31b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:27.161827 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdcfbaa-91a4-4b26-bbf5-856ef95e31b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:27.172227 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdcfbaa-91a4-4b26-bbf5-856ef95e31b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:27.173691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdcfbaa-91a4-4b26-bbf5-856ef95e31b9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:27.192170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-afdcfbaa-91a4-4b26-bbf5-856ef95e31b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:27.192170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:27.192170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:27.192170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:27.192170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:27.192170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:27.192170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:27.192170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:27.192170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:27.192170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:27.192170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:27.192170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:27.192170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:26:27.192170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:26:27.192170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:26:27.193245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:26:27.193245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:27.193245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:27.193245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:27.193245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:27.193245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:27.193245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:27.193245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:27.193245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:27.193245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:27.193245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:27.193245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:27.193245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:27.193245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:27.193245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:27.193245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:27.193245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:27.194175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:26:27.194175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:26:27.194175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:27.194175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:27.194175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 504/1045] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:26:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:26:27.200536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdcfbaa-91a4-4b26-bbf5-856ef95e31b9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:27.237031 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da7c46-5479-4c1f-91aa-fff7f334af9c None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:27.257562 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:27.272238 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca73cfc-416f-4282-ba34-30f68e3214d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:27.272914 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca73cfc-416f-4282-ba34-30f68e3214d1 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:27.273202 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca73cfc-416f-4282-ba34-30f68e3214d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:27.273482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca73cfc-416f-4282-ba34-30f68e3214d1 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:27.274197 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca73cfc-416f-4282-ba34-30f68e3214d1 None admin] RBAC: Adding query filter params (type=image) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:27.274509 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca73cfc-416f-4282-ba34-30f68e3214d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:27.275982 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca73cfc-416f-4282-ba34-30f68e3214d1 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:27.286755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 542/1046] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:26:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:27.287575 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca73cfc-416f-4282-ba34-30f68e3214d1 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:27.310026 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdcfbaa-91a4-4b26-bbf5-856ef95e31b9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:27.316996 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:27.346738 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07acd9f0-163b-41f8-bb77-4e18a9586226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:27.347407 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acd9f0-163b-41f8-bb77-4e18a9586226 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:27.347719 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acd9f0-163b-41f8-bb77-4e18a9586226 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:27.348002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acd9f0-163b-41f8-bb77-4e18a9586226 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:27.348735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07acd9f0-163b-41f8-bb77-4e18a9586226 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:27.349036 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07acd9f0-163b-41f8-bb77-4e18a9586226 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:27.350398 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07acd9f0-163b-41f8-bb77-4e18a9586226 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:27.362689 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 505/1047] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:27.367974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07acd9f0-163b-41f8-bb77-4e18a9586226 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:27.410539 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca73cfc-416f-4282-ba34-30f68e3214d1 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:27.454121 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da860e-587a-43e0-aebf-355146be0fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:27.454832 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da860e-587a-43e0-aebf-355146be0fab None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:27.455168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da860e-587a-43e0-aebf-355146be0fab None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:27.455443 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da860e-587a-43e0-aebf-355146be0fab None admin] PATH_INFO: `/v3/endpoints` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:27.456191 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da860e-587a-43e0-aebf-355146be0fab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:27.457571 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da860e-587a-43e0-aebf-355146be0fab None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:27.490677 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 543/1048] 10.222.0.67 () {60 vars in 1144 bytes} [Tue May 31 14:26:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:26:27.495952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07acd9f0-163b-41f8-bb77-4e18a9586226 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:27.511977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:27.543788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493be1b1-1882-4359-b5c8-1a8aadd045ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:27.544652 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493be1b1-1882-4359-b5c8-1a8aadd045ea None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:27.544948 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493be1b1-1882-4359-b5c8-1a8aadd045ea None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:27.545229 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493be1b1-1882-4359-b5c8-1a8aadd045ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:27.553307 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493be1b1-1882-4359-b5c8-1a8aadd045ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:27.554909 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493be1b1-1882-4359-b5c8-1a8aadd045ea None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:27.570611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-493be1b1-1882-4359-b5c8-1a8aadd045ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:26:27.570611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:27.570611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:27.570611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:27.570611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:27.570611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:27.570611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:27.570611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:27.570611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:27.570611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:27.570611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:27.570611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:27.570611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:26:27.570611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:26:27.570611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:26:27.572298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:26:27.572298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:27.572298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:27.572298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:27.572298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:27.572298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:27.572298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:27.572298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:27.572298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:27.572298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:27.572298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:27.572298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:27.572298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:27.572298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:27.572298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:27.572298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:27.572298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:27.573226 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:26:27.573226 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:26:27.573226 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:26:27.573226 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:26:27.573226 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:26:27.573226 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:26:27.573226 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:26:27.573226 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:27.574996 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 506/1049] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:26:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:26:27.578616 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da860e-587a-43e0-aebf-355146be0fab None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:27.668066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:27.691497 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da860e-587a-43e0-aebf-355146be0fab None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:27.727818 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24634dba-9bb5-48c6-9ac5-f09e6cdf7601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:27.728826 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24634dba-9bb5-48c6-9ac5-f09e6cdf7601 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:27.729111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24634dba-9bb5-48c6-9ac5-f09e6cdf7601 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:27.729383 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24634dba-9bb5-48c6-9ac5-f09e6cdf7601 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:27.730174 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24634dba-9bb5-48c6-9ac5-f09e6cdf7601 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:27.730485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24634dba-9bb5-48c6-9ac5-f09e6cdf7601 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:27.731966 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24634dba-9bb5-48c6-9ac5-f09e6cdf7601 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:27.745066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 544/1050] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:26:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:27.753693 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493be1b1-1882-4359-b5c8-1a8aadd045ea None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:27.853815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493be1b1-1882-4359-b5c8-1a8aadd045ea None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:27.888405 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4e2aa-7f10-4b49-85f4-9aab3afc22b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:27.889060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4e2aa-7f10-4b49-85f4-9aab3afc22b2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:27.889343 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4e2aa-7f10-4b49-85f4-9aab3afc22b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:27.889619 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4e2aa-7f10-4b49-85f4-9aab3afc22b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:27.890442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4e2aa-7f10-4b49-85f4-9aab3afc22b2 None admin] RBAC: Adding query filter params (role.id=8de91fffa27842109896bf18045db796, scope.project.id=f965a93a2f674598890bd8cafededb2c, user.id=68c6363b60954c3ebf50d2019f5f2fd2) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:27.890779 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4e2aa-7f10-4b49-85f4-9aab3afc22b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:27.892195 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4e2aa-7f10-4b49-85f4-9aab3afc22b2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:27.906739 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 507/1051] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:26:27 2022] GET /identity/v3/role_assignments?user.id=68c6363b60954c3ebf50d2019f5f2fd2&scope.project.id=f965a93a2f674598890bd8cafededb2c&role.id=8de91fffa27842109896bf18045db796 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:28.602941 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030f576-2932-4921-92a2-94c6258f9f64 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:28.603447 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030f576-2932-4921-92a2-94c6258f9f64 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:28.603862 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030f576-2932-4921-92a2-94c6258f9f64 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:28.605786 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 545/1052] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:28.611408 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d58ecc-bd80-407a-a4f6-c5201419ed11 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:28.611887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d58ecc-bd80-407a-a4f6-c5201419ed11 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:28.612079 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d58ecc-bd80-407a-a4f6-c5201419ed11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:28.631892 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-37d58ecc-bd80-407a-a4f6-c5201419ed11 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:28.633270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-37d58ecc-bd80-407a-a4f6-c5201419ed11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:28.661733 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:28.705442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 508/1053] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:28.729730 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030f576-2932-4921-92a2-94c6258f9f64 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:28.830356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030f576-2932-4921-92a2-94c6258f9f64 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:28.863260 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb95774-4a2a-4cf0-ad79-88114143f553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:28.863979 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb95774-4a2a-4cf0-ad79-88114143f553 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:28.864257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb95774-4a2a-4cf0-ad79-88114143f553 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:28.864531 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb95774-4a2a-4cf0-ad79-88114143f553 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:28.874772 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb95774-4a2a-4cf0-ad79-88114143f553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:26:28.882250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-3cb95774-4a2a-4cf0-ad79-88114143f553 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:26:28.882250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:28.882250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:28.882250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:28.882250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:28.882250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:28.882250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:28.882250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:28.882250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:28.882250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:28.882250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:28.882250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:28.882250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:26:28.882250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:26:28.882250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:26:28.882250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:26:28.883356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:26:28.883356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:26:28.883356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:26:28.883356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:26:28.883356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:28.883356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:28.883356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:28.883356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:28.883356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:28.883356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:28.883356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:28.883356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:26:28.883356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:28.883356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:28.883356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:28.883356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:28.884706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:28.884706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:28.884706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:28.884706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:28.884706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:26:28.884706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:26:28.884706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:26:28.884706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:26:28.884706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:26:28.884706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:26:28.884706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:26:28.884706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:28.886975 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 546/1054] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:26:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:28.890026 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d58ecc-bd80-407a-a4f6-c5201419ed11 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:28.988137 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d58ecc-bd80-407a-a4f6-c5201419ed11 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:29.022991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7daa286-9dfa-4cd7-92d9-a8a9f46fbf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:29.023664 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7daa286-9dfa-4cd7-92d9-a8a9f46fbf51 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:29.023945 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7daa286-9dfa-4cd7-92d9-a8a9f46fbf51 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:29.024216 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7daa286-9dfa-4cd7-92d9-a8a9f46fbf51 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:29.025023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7daa286-9dfa-4cd7-92d9-a8a9f46fbf51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:29.025327 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7daa286-9dfa-4cd7-92d9-a8a9f46fbf51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:29.026718 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7daa286-9dfa-4cd7-92d9-a8a9f46fbf51 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:29.039260 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 509/1055] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:26:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:29.041399 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb95774-4a2a-4cf0-ad79-88114143f553 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:29.044318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:29.150900 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb95774-4a2a-4cf0-ad79-88114143f553 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:29.185730 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc606db-8eab-416a-822b-cb191e990117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:29.186635 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc606db-8eab-416a-822b-cb191e990117 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:29.187003 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc606db-8eab-416a-822b-cb191e990117 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:29.187281 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc606db-8eab-416a-822b-cb191e990117 None admin] PATH_INFO: `/v3/users/5fbe12d725eb40879e0e09a3a1439b7a` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:29.201164 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc606db-8eab-416a-822b-cb191e990117 None admin] RBAC: Authorizing `identity:get_user(user_id=5fbe12d725eb40879e0e09a3a1439b7a)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:29.202627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc606db-8eab-416a-822b-cb191e990117 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:29.204466 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 547/1056] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:26:29 2022] GET /identity/v3/users/5fbe12d725eb40879e0e09a3a1439b7a => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:29.209974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7daa286-9dfa-4cd7-92d9-a8a9f46fbf51 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:29.237001 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:29.311305 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7daa286-9dfa-4cd7-92d9-a8a9f46fbf51 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:29.344457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fdab79-e7d6-4db1-adf3-0ca07c1d96c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:29.345112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdab79-e7d6-4db1-adf3-0ca07c1d96c8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:29.345395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdab79-e7d6-4db1-adf3-0ca07c1d96c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:29.345667 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdab79-e7d6-4db1-adf3-0ca07c1d96c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:29.411965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fdab79-e7d6-4db1-adf3-0ca07c1d96c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:29.413335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fdab79-e7d6-4db1-adf3-0ca07c1d96c8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:29.438706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:29.519809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 510/1057] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:26:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 31 14:26:29.525994 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc606db-8eab-416a-822b-cb191e990117 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:29.584302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:29.625919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc606db-8eab-416a-822b-cb191e990117 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:29.660882 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bc9bd-6774-4eb2-ba3e-32200b376ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:29.661744 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bc9bd-6774-4eb2-ba3e-32200b376ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:29.662073 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bc9bd-6774-4eb2-ba3e-32200b376ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:29.662294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bc9bd-6774-4eb2-ba3e-32200b376ee8 None admin] PATH_INFO: `/v3/projects/b72b6db8b7ee4acdbe588f86788a5c3e` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:29.663482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3bc9bd-6774-4eb2-ba3e-32200b376ee8 None admin] RBAC: Authorizing `identity:get_project(project_id=b72b6db8b7ee4acdbe588f86788a5c3e)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:29.664903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3bc9bd-6774-4eb2-ba3e-32200b376ee8 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:29.666844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 548/1058] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:26:29 2022] GET /identity/v3/projects/b72b6db8b7ee4acdbe588f86788a5c3e => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:29.672595 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fdab79-e7d6-4db1-adf3-0ca07c1d96c8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:29.777777 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fdab79-e7d6-4db1-adf3-0ca07c1d96c8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:29.784420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b4576-a054-410c-bc40-390a2c923e56 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:29.784715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b4576-a054-410c-bc40-390a2c923e56 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:29.784991 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b4576-a054-410c-bc40-390a2c923e56 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:29.787028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 549/1059] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:29.792041 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54487bf5-a51a-4193-b523-c004013efcac None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:29.792329 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54487bf5-a51a-4193-b523-c004013efcac None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:29.792659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54487bf5-a51a-4193-b523-c004013efcac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:29.807161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:29.810539 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-54487bf5-a51a-4193-b523-c004013efcac None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:29.812026 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-54487bf5-a51a-4193-b523-c004013efcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:29.821072 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81da8e-5bcc-4425-95e1-165abc9ae19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:29.822298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81da8e-5bcc-4425-95e1-165abc9ae19c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:29.822588 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81da8e-5bcc-4425-95e1-165abc9ae19c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:29.822903 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81da8e-5bcc-4425-95e1-165abc9ae19c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:29.823772 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81da8e-5bcc-4425-95e1-165abc9ae19c None admin] RBAC: Adding query filter params (role.id=e8d679596ffb45939229d1e2a13ce9b0, scope.project.id=b72b6db8b7ee4acdbe588f86788a5c3e, user.id=5fbe12d725eb40879e0e09a3a1439b7a) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:29.824093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81da8e-5bcc-4425-95e1-165abc9ae19c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:29.825849 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81da8e-5bcc-4425-95e1-165abc9ae19c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:29.842706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 511/1060] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:26:29 2022] GET /identity/v3/role_assignments?user.id=5fbe12d725eb40879e0e09a3a1439b7a&scope.project.id=b72b6db8b7ee4acdbe588f86788a5c3e&role.id=e8d679596ffb45939229d1e2a13ce9b0 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:29.900008 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 550/1061] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 31 14:26:29.923641 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81da8e-5bcc-4425-95e1-165abc9ae19c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:30.030424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81da8e-5bcc-4425-95e1-165abc9ae19c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:30.072873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68fcb0-b33a-4ce9-afec-548d4fcc5b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:30.073909 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68fcb0-b33a-4ce9-afec-548d4fcc5b79 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:30.074179 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68fcb0-b33a-4ce9-afec-548d4fcc5b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:30.074470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68fcb0-b33a-4ce9-afec-548d4fcc5b79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:30.137103 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:30.150423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68fcb0-b33a-4ce9-afec-548d4fcc5b79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:30.151958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68fcb0-b33a-4ce9-afec-548d4fcc5b79 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:30.153003 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9878a5-75f0-49d9-a475-ef9bd37abf29 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:30.153295 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9878a5-75f0-49d9-a475-ef9bd37abf29 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:30.153626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9878a5-75f0-49d9-a475-ef9bd37abf29 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:30.154985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 551/1062] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 31 14:26:30.159807 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d12f6-309e-45f3-a976-1270245463b1 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:30.160092 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d12f6-309e-45f3-a976-1270245463b1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:30.160374 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d12f6-309e-45f3-a976-1270245463b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:30.178568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-752d12f6-309e-45f3-a976-1270245463b1 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:30.179661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-752d12f6-309e-45f3-a976-1270245463b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:30.237697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:30.254425 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 552/1063] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 31 14:26:30.261706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 512/1064] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:26:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 31 14:26:30.262889 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d12f6-309e-45f3-a976-1270245463b1 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:30.278546 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68fcb0-b33a-4ce9-afec-548d4fcc5b79 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:30.369186 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d12f6-309e-45f3-a976-1270245463b1 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:30.369748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:30.382246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68fcb0-b33a-4ce9-afec-548d4fcc5b79 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:30.407924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4314c-5eb5-4d5b-8dc7-f7c06fe4b086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:30.408851 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4314c-5eb5-4d5b-8dc7-f7c06fe4b086 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:30.409131 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4314c-5eb5-4d5b-8dc7-f7c06fe4b086 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:30.409520 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4314c-5eb5-4d5b-8dc7-f7c06fe4b086 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:30.419101 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b94c34-c6ae-4960-a7e7-83ba7e3dde44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:30.420324 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b94c34-c6ae-4960-a7e7-83ba7e3dde44 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:30.420613 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b94c34-c6ae-4960-a7e7-83ba7e3dde44 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:30.421074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b94c34-c6ae-4960-a7e7-83ba7e3dde44 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:30.421826 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4314c-5eb5-4d5b-8dc7-f7c06fe4b086 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:30.423448 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4314c-5eb5-4d5b-8dc7-f7c06fe4b086 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:30.432073 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b94c34-c6ae-4960-a7e7-83ba7e3dde44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:26:30.440182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f2b94c34-c6ae-4960-a7e7-83ba7e3dde44 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 31 14:26:30.440182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:30.440182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:30.440182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:30.440182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:30.440182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:30.440182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:30.440182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:30.440182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:30.440182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:30.440182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:30.440182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:30.440182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:26:30.440182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:26:30.440182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:26:30.440182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:26:30.441203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:26:30.441203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:26:30.441203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:26:30.441203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:26:30.441203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:30.441203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:30.441203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:30.441203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:30.441203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:30.441203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:30.441203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:30.441203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:30.441203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:30.441203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:30.441203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:30.441203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:30.448018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:30.448018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:30.448018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:30.448018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:30.448018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:26:30.448018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:26:30.448018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:26:30.448018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:26:30.448018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:26:30.448018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:26:30.448018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 31 14:26:30.448018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:30.448018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 513/1065] 10.222.0.67 () {56 vars in 1120 bytes} [Tue May 31 14:26:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:26:30.448791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-81f4314c-5eb5-4d5b-8dc7-f7c06fe4b086 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:30.448791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:30.448791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:30.448791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:30.448791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:30.448791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:30.448791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:30.448791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:30.448791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:30.448791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:30.448791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:30.448791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:30.448791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:26:30.448791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:26:30.448791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:26:30.449689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:26:30.449689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:30.449689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:30.449689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:30.449689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:30.449689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:30.449689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:30.449689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:30.449689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:26:30.449689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:30.449689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:30.449689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:30.449689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:30.449689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:30.449689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:30.449689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:30.449689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:30.452451 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:26:30.452451 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:26:30.452451 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:30.452451 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:30.452451 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 553/1066] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:26:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:30.452842 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b94c34-c6ae-4960-a7e7-83ba7e3dde44 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:30.452936 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4314c-5eb5-4d5b-8dc7-f7c06fe4b086 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:30.495829 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:30.557002 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4314c-5eb5-4d5b-8dc7-f7c06fe4b086 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:30.559228 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b94c34-c6ae-4960-a7e7-83ba7e3dde44 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:30.595097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c760b47-33ec-4a7d-9850-0f88dbe10212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:30.595805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c760b47-33ec-4a7d-9850-0f88dbe10212 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:30.596056 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c760b47-33ec-4a7d-9850-0f88dbe10212 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:30.596329 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c760b47-33ec-4a7d-9850-0f88dbe10212 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:30.597083 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c760b47-33ec-4a7d-9850-0f88dbe10212 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:30.597392 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c760b47-33ec-4a7d-9850-0f88dbe10212 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:30.598784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c760b47-33ec-4a7d-9850-0f88dbe10212 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:30.600041 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce7390-2541-4c62-9978-6ef6754621a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:30.600814 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce7390-2541-4c62-9978-6ef6754621a2 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:30.601303 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce7390-2541-4c62-9978-6ef6754621a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:30.601427 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce7390-2541-4c62-9978-6ef6754621a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:30.602075 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ce7390-2541-4c62-9978-6ef6754621a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:30.602380 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ce7390-2541-4c62-9978-6ef6754621a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:30.604367 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ce7390-2541-4c62-9978-6ef6754621a2 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:30.608096 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 514/1067] 10.222.0.67 () {56 vars in 1115 bytes} [Tue May 31 14:26:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:30.613518 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c760b47-33ec-4a7d-9850-0f88dbe10212 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:30.626674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 554/1068] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:26:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:30.637752 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:30.735383 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c760b47-33ec-4a7d-9850-0f88dbe10212 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:30.770424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b78428-2a2d-40e0-a34e-e06b96250874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:30.771269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b78428-2a2d-40e0-a34e-e06b96250874 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:30.771557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b78428-2a2d-40e0-a34e-e06b96250874 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:30.771832 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b78428-2a2d-40e0-a34e-e06b96250874 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:30.784119 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b78428-2a2d-40e0-a34e-e06b96250874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:30.785592 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b78428-2a2d-40e0-a34e-e06b96250874 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:30.804279 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-70b78428-2a2d-40e0-a34e-e06b96250874 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:30.804279 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:30.804279 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:30.804279 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:30.804279 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:30.804279 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:30.804279 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:30.804279 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:30.804279 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:30.804279 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:30.804279 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:30.804279 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:30.804279 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:26:30.804279 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:26:30.804279 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:26:30.805270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:26:30.805270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:30.805270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:30.805270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:30.805270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:30.805270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:30.805270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:30.805270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:30.805270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:30.805270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:30.805270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:30.805270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:30.805270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:30.805270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:30.805270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:30.805270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:30.805270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:30.806244 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:26:30.806244 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:26:30.806244 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:30.806244 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:30.810846 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 515/1069] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:26:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:26:30.812036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce7390-2541-4c62-9978-6ef6754621a2 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:30.920594 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:30.923869 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce7390-2541-4c62-9978-6ef6754621a2 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:30.960199 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35ea24-2a89-4a4a-a779-99d624aef61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:30.960890 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35ea24-2a89-4a4a-a779-99d624aef61a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:30.961173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35ea24-2a89-4a4a-a779-99d624aef61a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:30.961451 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35ea24-2a89-4a4a-a779-99d624aef61a None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:30.962752 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35ea24-2a89-4a4a-a779-99d624aef61a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:30.963071 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35ea24-2a89-4a4a-a779-99d624aef61a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:30.964479 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35ea24-2a89-4a4a-a779-99d624aef61a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:30.978602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 555/1070] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:26:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:30.984157 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b78428-2a2d-40e0-a34e-e06b96250874 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:31.045184 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:31.084845 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b78428-2a2d-40e0-a34e-e06b96250874 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:31.119353 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade7a9e-38f6-4db4-b944-8ceb9703a3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:31.120160 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade7a9e-38f6-4db4-b944-8ceb9703a3df None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:31.120446 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade7a9e-38f6-4db4-b944-8ceb9703a3df None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:31.120724 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade7a9e-38f6-4db4-b944-8ceb9703a3df None admin] PATH_INFO: `/v3/users/nova` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:31.128573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade7a9e-38f6-4db4-b944-8ceb9703a3df None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:31.130008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade7a9e-38f6-4db4-b944-8ceb9703a3df None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:31.144209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-bade7a9e-38f6-4db4-b944-8ceb9703a3df None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 31 14:26:31.144209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:31.144209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:31.144209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:31.144209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:31.144209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:31.144209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:31.144209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:31.144209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:31.144209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:31.144209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:31.144209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:31.144209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:26:31.144209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:26:31.144209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:26:31.144209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:26:31.145352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:31.145352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:31.145352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:26:31.145352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:26:31.145352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:26:31.145352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:26:31.145352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:31.145352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:31.145352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:31.145352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:31.145352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:31.145352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:31.145352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:31.145352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:31.145352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:31.145352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:31.145352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:31.145352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:31.151748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:31.151748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:31.151748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:26:31.151748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:26:31.151748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:26:31.151748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:26:31.151748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:26:31.151748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:26:31.151748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 31 14:26:31.151748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:31.151748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 516/1071] 10.222.0.67 () {56 vars in 1093 bytes} [Tue May 31 14:26:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:31.152380 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35ea24-2a89-4a4a-a779-99d624aef61a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:31.246706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35ea24-2a89-4a4a-a779-99d624aef61a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:31.279453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d025a4-166f-4349-900e-fbb080b9c4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:31.280113 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d025a4-166f-4349-900e-fbb080b9c4eb None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:31.280397 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d025a4-166f-4349-900e-fbb080b9c4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:31.280673 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d025a4-166f-4349-900e-fbb080b9c4eb None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:31.281497 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d025a4-166f-4349-900e-fbb080b9c4eb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:31.281805 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d025a4-166f-4349-900e-fbb080b9c4eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:31.283263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d025a4-166f-4349-900e-fbb080b9c4eb None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:31.300035 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 556/1072] 10.222.0.67 () {56 vars in 1133 bytes} [Tue May 31 14:26:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:31.305324 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade7a9e-38f6-4db4-b944-8ceb9703a3df None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:31.407444 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade7a9e-38f6-4db4-b944-8ceb9703a3df None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:31.428849 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:31.441276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c4cee-8690-4709-9aec-77b984857101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:31.442069 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c4cee-8690-4709-9aec-77b984857101 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:31.442351 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c4cee-8690-4709-9aec-77b984857101 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:31.442625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c4cee-8690-4709-9aec-77b984857101 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:31.453785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c4cee-8690-4709-9aec-77b984857101 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:31.455530 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c4cee-8690-4709-9aec-77b984857101 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:31.471866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-863c4cee-8690-4709-9aec-77b984857101 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:31.471866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:31.471866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:31.471866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:31.471866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:31.471866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:31.471866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:31.471866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:31.471866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:31.471866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:31.471866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:31.471866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:31.471866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:26:31.471866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:26:31.471866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:26:31.472731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:26:31.472731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:31.472731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:31.472731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:31.472731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:31.472731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:31.472731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:31.472731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:31.472731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:31.472731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:31.472731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:31.472731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:31.472731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:31.472731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:31.472731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:31.472731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:31.472731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:31.473636 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:26:31.473636 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:26:31.473636 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:31.473636 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:31.473636 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 517/1073] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:26:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:31.478656 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d025a4-166f-4349-900e-fbb080b9c4eb None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:31.497626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:31.577146 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d025a4-166f-4349-900e-fbb080b9c4eb None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:31.609533 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71a474-2e14-45e5-be24-7a4a352169c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:31.610191 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71a474-2e14-45e5-be24-7a4a352169c1 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:31.610475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71a474-2e14-45e5-be24-7a4a352169c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:31.610784 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71a474-2e14-45e5-be24-7a4a352169c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:31.611505 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71a474-2e14-45e5-be24-7a4a352169c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:31.611813 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71a474-2e14-45e5-be24-7a4a352169c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:31.613183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71a474-2e14-45e5-be24-7a4a352169c1 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:31.624703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 557/1074] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:31.630314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c4cee-8690-4709-9aec-77b984857101 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:31.726415 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c4cee-8690-4709-9aec-77b984857101 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:31.759700 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4b53a-615d-4082-8f30-b7d610fcd8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:31.760517 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4b53a-615d-4082-8f30-b7d610fcd8b8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:31.760798 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4b53a-615d-4082-8f30-b7d610fcd8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:31.761073 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4b53a-615d-4082-8f30-b7d610fcd8b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:31.768754 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4b53a-615d-4082-8f30-b7d610fcd8b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:31.770218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4b53a-615d-4082-8f30-b7d610fcd8b8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:31.785385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-13a4b53a-615d-4082-8f30-b7d610fcd8b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:26:31.785385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:31.785385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:31.785385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:31.785385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:31.785385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:31.785385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:31.785385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:31.785385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:31.785385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:31.785385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:31.785385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:31.785385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:26:31.785385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:26:31.785385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:26:31.786540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:26:31.786540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:31.786540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:31.786540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:31.786540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:31.786540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:31.786540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:31.786540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:31.786540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:31.786540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:31.786540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:31.786540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:31.786540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:31.786540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:31.786540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:31.786540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:31.786540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:31.787488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:26:31.787488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:26:31.787488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:26:31.787488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:26:31.787488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:26:31.787488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:26:31.787488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:26:31.787488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:31.787488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 518/1075] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:26:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:26:31.793172 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71a474-2e14-45e5-be24-7a4a352169c1 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:31.869915 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:31.891367 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71a474-2e14-45e5-be24-7a4a352169c1 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:31.925028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de551bf7-d648-4433-aed6-e9fd4f86e875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:31.925689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de551bf7-d648-4433-aed6-e9fd4f86e875 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:31.925994 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de551bf7-d648-4433-aed6-e9fd4f86e875 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:31.926272 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de551bf7-d648-4433-aed6-e9fd4f86e875 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:31.927084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de551bf7-d648-4433-aed6-e9fd4f86e875 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:31.927395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de551bf7-d648-4433-aed6-e9fd4f86e875 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:31.928825 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de551bf7-d648-4433-aed6-e9fd4f86e875 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:31.941205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 558/1076] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:26:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:31.947029 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4b53a-615d-4082-8f30-b7d610fcd8b8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:31.980289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:32.044418 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4b53a-615d-4082-8f30-b7d610fcd8b8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:32.077236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3ec3b-2917-48c4-880f-a73384325af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:32.078118 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3ec3b-2917-48c4-880f-a73384325af7 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:32.078399 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3ec3b-2917-48c4-880f-a73384325af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:32.078705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3ec3b-2917-48c4-880f-a73384325af7 None admin] PATH_INFO: `/v3/projects/f965a93a2f674598890bd8cafededb2c/users/68c6363b60954c3ebf50d2019f5f2fd2/roles/8de91fffa27842109896bf18045db796` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:32.080865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3ec3b-2917-48c4-880f-a73384325af7 None admin] RBAC: Authorizing `identity:create_grant(project_id=f965a93a2f674598890bd8cafededb2c, user_id=68c6363b60954c3ebf50d2019f5f2fd2, role_id=8de91fffa27842109896bf18045db796)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:32.082270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3ec3b-2917-48c4-880f-a73384325af7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:32.094030 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580aca9-c170-405d-88e8-f1c78ce53c67 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:32.094344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580aca9-c170-405d-88e8-f1c78ce53c67 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:32.094623 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580aca9-c170-405d-88e8-f1c78ce53c67 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:32.095607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 519/1077] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:26:31 2022] PUT /identity/v3/projects/f965a93a2f674598890bd8cafededb2c/users/68c6363b60954c3ebf50d2019f5f2fd2/roles/8de91fffa27842109896bf18045db796 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:26:32.096000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 559/1078] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:32.101668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e3072-8aea-43a8-a506-f5761d35e709 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:32.101957 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e3072-8aea-43a8-a506-f5761d35e709 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:32.102234 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e3072-8aea-43a8-a506-f5761d35e709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:32.120536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7d3e3072-8aea-43a8-a506-f5761d35e709 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:32.121672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7d3e3072-8aea-43a8-a506-f5761d35e709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:32.192344 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 520/1079] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:32.215665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580aca9-c170-405d-88e8-f1c78ce53c67 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:32.321660 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580aca9-c170-405d-88e8-f1c78ce53c67 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:32.355570 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a92b0-b439-460b-96d4-830837fd8660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:32.356368 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a92b0-b439-460b-96d4-830837fd8660 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:32.356653 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a92b0-b439-460b-96d4-830837fd8660 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:32.356929 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a92b0-b439-460b-96d4-830837fd8660 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:32.363628 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:32.367459 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2a92b0-b439-460b-96d4-830837fd8660 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:26:32.375499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-4c2a92b0-b439-460b-96d4-830837fd8660 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:26:32.375499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:32.375499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:32.375499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:32.375499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:32.375499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:32.375499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:32.375499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:32.375499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:32.375499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:32.375499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:32.375499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:32.375499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:26:32.375499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:26:32.375499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:26:32.375499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:26:32.376564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:26:32.376564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:26:32.376564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:26:32.376564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:26:32.376564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:32.376564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:32.376564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:32.376564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:32.376564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:32.376564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:32.376564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:32.376564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:26:32.376564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:32.376564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:32.376564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:32.376564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:32.377432 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:32.377432 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:32.377432 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:32.377432 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:32.377432 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:26:32.377432 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:26:32.377432 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:26:32.377432 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:26:32.377432 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:26:32.377432 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:26:32.377432 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:26:32.377432 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:32.377432 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 560/1080] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:26:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:32.382777 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e3072-8aea-43a8-a506-f5761d35e709 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:32.407145 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:32.481717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e3072-8aea-43a8-a506-f5761d35e709 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:32.514545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0db6ae-f041-46ae-b631-c00884462ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:32.515246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0db6ae-f041-46ae-b631-c00884462ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:32.515535 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0db6ae-f041-46ae-b631-c00884462ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:32.515813 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0db6ae-f041-46ae-b631-c00884462ba3 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:32.516560 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0db6ae-f041-46ae-b631-c00884462ba3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:32.516874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0db6ae-f041-46ae-b631-c00884462ba3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:32.518239 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0db6ae-f041-46ae-b631-c00884462ba3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:32.526792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 521/1081] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:26:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:32.532600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a92b0-b439-460b-96d4-830837fd8660 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:32.630317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a92b0-b439-460b-96d4-830837fd8660 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:32.662751 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2054b3a-b152-4eca-8da0-e6ad489026bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:32.663619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2054b3a-b152-4eca-8da0-e6ad489026bc None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:32.663897 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2054b3a-b152-4eca-8da0-e6ad489026bc None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:32.664168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2054b3a-b152-4eca-8da0-e6ad489026bc None admin] PATH_INFO: `/v3/users/5fbe12d725eb40879e0e09a3a1439b7a` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:32.665358 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2054b3a-b152-4eca-8da0-e6ad489026bc None admin] RBAC: Authorizing `identity:get_user(user_id=5fbe12d725eb40879e0e09a3a1439b7a)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:32.666817 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2054b3a-b152-4eca-8da0-e6ad489026bc None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:32.668436 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 561/1082] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:26:32 2022] GET /identity/v3/users/5fbe12d725eb40879e0e09a3a1439b7a => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:32.674247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0db6ae-f041-46ae-b631-c00884462ba3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:32.758350 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:32.774100 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0db6ae-f041-46ae-b631-c00884462ba3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:32.806901 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e9fa5-6b31-4b7a-8024-e99d877e5e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:32.807798 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e9fa5-6b31-4b7a-8024-e99d877e5e35 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:32.808087 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e9fa5-6b31-4b7a-8024-e99d877e5e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:32.808369 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e9fa5-6b31-4b7a-8024-e99d877e5e35 None admin] PATH_INFO: `/v3/projects/b72b6db8b7ee4acdbe588f86788a5c3e` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:32.809575 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e9fa5-6b31-4b7a-8024-e99d877e5e35 None admin] RBAC: Authorizing `identity:get_project(project_id=b72b6db8b7ee4acdbe588f86788a5c3e)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:32.811040 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e9fa5-6b31-4b7a-8024-e99d877e5e35 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:32.812755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 522/1083] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:26:32 2022] GET /identity/v3/projects/b72b6db8b7ee4acdbe588f86788a5c3e => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:32.818696 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2054b3a-b152-4eca-8da0-e6ad489026bc None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:32.882257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:32.917777 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2054b3a-b152-4eca-8da0-e6ad489026bc None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:32.951333 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f60e3a4-2941-46be-b1ae-bad5ea1f9c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:32.952251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60e3a4-2941-46be-b1ae-bad5ea1f9c43 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:32.952537 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60e3a4-2941-46be-b1ae-bad5ea1f9c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:32.952813 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60e3a4-2941-46be-b1ae-bad5ea1f9c43 None admin] PATH_INFO: `/v3/projects/b72b6db8b7ee4acdbe588f86788a5c3e/users/5fbe12d725eb40879e0e09a3a1439b7a/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:32.954790 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f60e3a4-2941-46be-b1ae-bad5ea1f9c43 None admin] RBAC: Authorizing `identity:create_grant(project_id=b72b6db8b7ee4acdbe588f86788a5c3e, user_id=5fbe12d725eb40879e0e09a3a1439b7a, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:32.956224 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f60e3a4-2941-46be-b1ae-bad5ea1f9c43 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:32.968816 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 562/1084] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:26:32 2022] PUT /identity/v3/projects/b72b6db8b7ee4acdbe588f86788a5c3e/users/5fbe12d725eb40879e0e09a3a1439b7a/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:26:34.304196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88e1fc-f331-4328-a314-e0974148aa91 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:34.304741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88e1fc-f331-4328-a314-e0974148aa91 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:34.304741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88e1fc-f331-4328-a314-e0974148aa91 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:34.306123 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 523/1085] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:34.311480 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f819339-20b5-4992-b4d7-419d3d3ccd72 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:34.311895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f819339-20b5-4992-b4d7-419d3d3ccd72 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:34.312046 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f819339-20b5-4992-b4d7-419d3d3ccd72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:34.329543 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-3f819339-20b5-4992-b4d7-419d3d3ccd72 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:34.330777 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3f819339-20b5-4992-b4d7-419d3d3ccd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:34.395631 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 563/1086] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:34.418858 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88e1fc-f331-4328-a314-e0974148aa91 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:34.518867 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88e1fc-f331-4328-a314-e0974148aa91 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:34.551046 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e347bb-6efb-4a85-98e8-b39257097b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:34.551860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e347bb-6efb-4a85-98e8-b39257097b3d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:34.552142 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e347bb-6efb-4a85-98e8-b39257097b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:34.552416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e347bb-6efb-4a85-98e8-b39257097b3d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:34.563283 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e347bb-6efb-4a85-98e8-b39257097b3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:26:34.571946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-d0e347bb-6efb-4a85-98e8-b39257097b3d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 31 14:26:34.571946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:34.571946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:34.571946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:34.571946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:34.571946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:34.571946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:34.571946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:34.571946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:34.571946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:34.571946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:34.571946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:34.571946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:26:34.571946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:26:34.571946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:26:34.571946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:26:34.572879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:26:34.572879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:26:34.572879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:26:34.572879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:26:34.572879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:34.572879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:34.572879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:34.572879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:34.572879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:34.572879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:34.572879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:34.572879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:34.572879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:34.572879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:34.572879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:34.572879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:34.573722 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:34.573722 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:34.573722 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:34.573722 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:34.573722 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:26:34.573722 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:26:34.573722 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:26:34.573722 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:26:34.573722 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:26:34.573722 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:26:34.573722 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 31 14:26:34.573722 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:34.573722 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 524/1087] 10.222.0.67 () {56 vars in 1120 bytes} [Tue May 31 14:26:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:34.579437 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f819339-20b5-4992-b4d7-419d3d3ccd72 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:34.633695 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:34.677520 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f819339-20b5-4992-b4d7-419d3d3ccd72 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:34.711520 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e51434-6b94-4d04-ad83-a69a0ae94d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:34.712250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e51434-6b94-4d04-ad83-a69a0ae94d88 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:34.712533 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e51434-6b94-4d04-ad83-a69a0ae94d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:34.712813 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e51434-6b94-4d04-ad83-a69a0ae94d88 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:34.713562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e51434-6b94-4d04-ad83-a69a0ae94d88 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:34.713877 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e51434-6b94-4d04-ad83-a69a0ae94d88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:34.715285 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e51434-6b94-4d04-ad83-a69a0ae94d88 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:34.728267 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 564/1088] 10.222.0.67 () {56 vars in 1115 bytes} [Tue May 31 14:26:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:34.729724 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e347bb-6efb-4a85-98e8-b39257097b3d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:34.746805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:34.827925 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e347bb-6efb-4a85-98e8-b39257097b3d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:34.864600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d5b14-ba76-4312-8e14-f3a4f7367560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:34.865404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d5b14-ba76-4312-8e14-f3a4f7367560 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:34.865690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d5b14-ba76-4312-8e14-f3a4f7367560 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:34.865968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d5b14-ba76-4312-8e14-f3a4f7367560 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:34.876862 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d5b14-ba76-4312-8e14-f3a4f7367560 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:34.878296 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d5b14-ba76-4312-8e14-f3a4f7367560 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:34.895379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-829d5b14-ba76-4312-8e14-f3a4f7367560 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:34.895379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:34.895379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:34.895379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:34.895379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:34.895379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:34.895379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:34.895379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:34.895379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:34.895379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:34.895379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:34.895379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:34.895379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:26:34.895379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:26:34.895379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:26:34.896442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:26:34.896442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:34.896442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:34.896442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:34.896442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:34.896442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:34.896442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:34.896442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:34.896442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:34.896442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:34.896442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:34.896442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:34.896442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:34.896442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:34.896442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:34.896442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:34.896442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:34.897389 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:26:34.897389 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:26:34.897389 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:34.897389 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:34.897389 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 525/1089] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:26:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:34.902341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e51434-6b94-4d04-ad83-a69a0ae94d88 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:34.999462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e51434-6b94-4d04-ad83-a69a0ae94d88 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:35.015945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:35.033240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4dc0c2-d4ba-4ad0-be1e-7f7f2372a425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:35.033901 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4dc0c2-d4ba-4ad0-be1e-7f7f2372a425 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:35.034187 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4dc0c2-d4ba-4ad0-be1e-7f7f2372a425 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:35.034465 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4dc0c2-d4ba-4ad0-be1e-7f7f2372a425 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:35.035263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4dc0c2-d4ba-4ad0-be1e-7f7f2372a425 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:35.035536 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4dc0c2-d4ba-4ad0-be1e-7f7f2372a425 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:35.036898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4dc0c2-d4ba-4ad0-be1e-7f7f2372a425 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:35.049040 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 565/1090] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:26:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:35.054614 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d5b14-ba76-4312-8e14-f3a4f7367560 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:35.122192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:35.156297 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d5b14-ba76-4312-8e14-f3a4f7367560 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:35.165709 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851b526-2324-4f61-93ef-5e03e4bd3bd0 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:35.166004 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851b526-2324-4f61-93ef-5e03e4bd3bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:35.166281 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851b526-2324-4f61-93ef-5e03e4bd3bd0 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:35.167699 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 566/1091] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 31 14:26:35.172619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e3e1c-b219-4d9d-b956-3fe657f6ed69 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:35.172907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e3e1c-b219-4d9d-b956-3fe657f6ed69 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:35.173185 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e3e1c-b219-4d9d-b956-3fe657f6ed69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:35.189371 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-2a0e3e1c-b219-4d9d-b956-3fe657f6ed69 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:35.190563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2a0e3e1c-b219-4d9d-b956-3fe657f6ed69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:35.191066 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c1a69-9392-4b31-998b-01e639ab77c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:35.191862 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c1a69-9392-4b31-998b-01e639ab77c7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:35.192150 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c1a69-9392-4b31-998b-01e639ab77c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:35.192434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c1a69-9392-4b31-998b-01e639ab77c7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:35.201183 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c1a69-9392-4b31-998b-01e639ab77c7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:35.202625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c1a69-9392-4b31-998b-01e639ab77c7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:35.216876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-132c1a69-9392-4b31-998b-01e639ab77c7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 31 14:26:35.216876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:35.216876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:35.216876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:35.216876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:35.216876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:35.216876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:35.216876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:35.216876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:35.216876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:35.216876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:35.216876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:35.216876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:26:35.216876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:26:35.216876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:26:35.216876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:26:35.217837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:35.217837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:35.217837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:26:35.217837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:26:35.217837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:26:35.217837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:26:35.217837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:35.217837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:35.217837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:35.217837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:35.217837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:35.217837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:35.217837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:35.217837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:35.217837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:35.217837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:35.217837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:35.217837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:35.218854 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:35.218854 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:35.218854 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:26:35.218854 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:26:35.218854 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:26:35.218854 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:26:35.218854 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:26:35.218854 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:26:35.218854 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 31 14:26:35.218854 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:35.218854 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 526/1092] 10.222.0.67 () {56 vars in 1093 bytes} [Tue May 31 14:26:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:35.223654 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c1a69-9392-4b31-998b-01e639ab77c7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:35.258162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 567/1093] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 31 14:26:35.281743 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e3e1c-b219-4d9d-b956-3fe657f6ed69 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:35.321120 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c1a69-9392-4b31-998b-01e639ab77c7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:35.346954 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:35.357405 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de788594-9f56-40a2-b8b4-92929a7f09d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:35.358069 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de788594-9f56-40a2-b8b4-92929a7f09d9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:35.358351 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de788594-9f56-40a2-b8b4-92929a7f09d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:35.358625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de788594-9f56-40a2-b8b4-92929a7f09d9 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:35.359488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de788594-9f56-40a2-b8b4-92929a7f09d9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:35.359791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de788594-9f56-40a2-b8b4-92929a7f09d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:35.361196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de788594-9f56-40a2-b8b4-92929a7f09d9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:35.380389 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 527/1094] 10.222.0.67 () {56 vars in 1133 bytes} [Tue May 31 14:26:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:35.382935 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e3e1c-b219-4d9d-b956-3fe657f6ed69 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:35.386092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de788594-9f56-40a2-b8b4-92929a7f09d9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:35.392322 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:35.417777 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea80e08-09a2-415d-bd0b-0fe5c3672297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:35.418566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea80e08-09a2-415d-bd0b-0fe5c3672297 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:35.418879 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea80e08-09a2-415d-bd0b-0fe5c3672297 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:35.419154 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea80e08-09a2-415d-bd0b-0fe5c3672297 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:35.429152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea80e08-09a2-415d-bd0b-0fe5c3672297 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:26:35.436779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-eea80e08-09a2-415d-bd0b-0fe5c3672297 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:26:35.436779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:35.436779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:35.436779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:35.436779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:35.436779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:35.436779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:35.436779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:35.436779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:35.436779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:35.436779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:35.436779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:35.436779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:26:35.436779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:26:35.436779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:26:35.436779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:26:35.437828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:26:35.437828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:26:35.437828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:26:35.437828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:26:35.437828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:35.437828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:35.437828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:35.437828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:35.437828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:35.437828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:35.437828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:35.437828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:26:35.437828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:35.437828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:35.437828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:35.437828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:35.438720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:35.438720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:35.438720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:35.438720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:35.438720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:26:35.438720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:26:35.438720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:26:35.438720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:26:35.438720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:26:35.438720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:26:35.438720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:26:35.438720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:35.438720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 568/1095] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:26:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:26:35.443795 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea80e08-09a2-415d-bd0b-0fe5c3672297 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:35.485727 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de788594-9f56-40a2-b8b4-92929a7f09d9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:35.518907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f3220-3e0e-46f4-80a9-c106bf8039f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:35.519492 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f3220-3e0e-46f4-80a9-c106bf8039f0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:35.519773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f3220-3e0e-46f4-80a9-c106bf8039f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:35.520047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f3220-3e0e-46f4-80a9-c106bf8039f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:35.542435 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea80e08-09a2-415d-bd0b-0fe5c3672297 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:35.569153 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:35.576323 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87febbb-5ac6-4f44-8d24-270c2b8afa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:35.576987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87febbb-5ac6-4f44-8d24-270c2b8afa3f None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:35.577276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87febbb-5ac6-4f44-8d24-270c2b8afa3f None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:35.577553 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87febbb-5ac6-4f44-8d24-270c2b8afa3f None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:35.578296 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87febbb-5ac6-4f44-8d24-270c2b8afa3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:35.578607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87febbb-5ac6-4f44-8d24-270c2b8afa3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:35.580022 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87febbb-5ac6-4f44-8d24-270c2b8afa3f None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:35.585765 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f3220-3e0e-46f4-80a9-c106bf8039f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:35.587760 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f3220-3e0e-46f4-80a9-c106bf8039f0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:35.588612 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 569/1096] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:26:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:35.594274 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87febbb-5ac6-4f44-8d24-270c2b8afa3f None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:35.623052 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:35.684370 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 528/1097] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:26:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 31 14:26:35.689990 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f3220-3e0e-46f4-80a9-c106bf8039f0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:35.691239 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87febbb-5ac6-4f44-8d24-270c2b8afa3f None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:35.724625 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875366f-4db6-4de1-aaad-115dc3fa8a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:35.725487 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875366f-4db6-4de1-aaad-115dc3fa8a34 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:35.725767 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875366f-4db6-4de1-aaad-115dc3fa8a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:35.726095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875366f-4db6-4de1-aaad-115dc3fa8a34 None admin] PATH_INFO: `/v3/users/5fbe12d725eb40879e0e09a3a1439b7a` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:35.727315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0875366f-4db6-4de1-aaad-115dc3fa8a34 None admin] RBAC: Authorizing `identity:get_user(user_id=5fbe12d725eb40879e0e09a3a1439b7a)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:35.728773 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0875366f-4db6-4de1-aaad-115dc3fa8a34 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:35.730388 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 570/1098] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:26:35 2022] GET /identity/v3/users/5fbe12d725eb40879e0e09a3a1439b7a => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:35.735993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875366f-4db6-4de1-aaad-115dc3fa8a34 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:35.771413 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:35.787714 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f3220-3e0e-46f4-80a9-c106bf8039f0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:35.821912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe41849-95b6-48ef-b9e8-691b8e41c965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:35.822726 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe41849-95b6-48ef-b9e8-691b8e41c965 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:35.823015 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe41849-95b6-48ef-b9e8-691b8e41c965 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:35.823296 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe41849-95b6-48ef-b9e8-691b8e41c965 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:35.829554 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:35.833502 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe41849-95b6-48ef-b9e8-691b8e41c965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:35.833752 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875366f-4db6-4de1-aaad-115dc3fa8a34 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:35.834973 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe41849-95b6-48ef-b9e8-691b8e41c965 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:35.852579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-5fe41849-95b6-48ef-b9e8-691b8e41c965 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:35.852579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:35.852579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:35.852579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:35.852579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:35.852579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:35.852579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:35.852579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:35.852579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:35.852579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:35.852579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:35.852579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:35.852579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:26:35.852579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:26:35.852579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:26:35.853625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:26:35.853625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:35.853625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:35.853625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:35.853625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:35.853625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:35.853625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:35.853625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:35.853625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:35.853625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:35.853625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:35.853625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:35.853625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:35.853625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:35.853625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:35.853625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:35.853625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:35.854694 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:26:35.854694 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:26:35.854694 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:26:35.854694 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:35.854694 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 529/1099] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:26:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:26:35.859308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe41849-95b6-48ef-b9e8-691b8e41c965 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:35.867310 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552b36f-fdce-4484-aeb0-87c8b76b2b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:35.867958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552b36f-fdce-4484-aeb0-87c8b76b2b61 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:35.868764 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552b36f-fdce-4484-aeb0-87c8b76b2b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:35.869039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552b36f-fdce-4484-aeb0-87c8b76b2b61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:35.937595 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552b36f-fdce-4484-aeb0-87c8b76b2b61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:35.939093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552b36f-fdce-4484-aeb0-87c8b76b2b61 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:35.957471 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe41849-95b6-48ef-b9e8-691b8e41c965 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:35.990798 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b58121-9b17-4569-8d79-6fa63785d921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:35.991453 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b58121-9b17-4569-8d79-6fa63785d921 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:35.991735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b58121-9b17-4569-8d79-6fa63785d921 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:35.992011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b58121-9b17-4569-8d79-6fa63785d921 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:35.992719 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b58121-9b17-4569-8d79-6fa63785d921 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:35.993024 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b58121-9b17-4569-8d79-6fa63785d921 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:35.994919 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b58121-9b17-4569-8d79-6fa63785d921 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:35.997432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:36.006530 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 530/1100] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:36.012640 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b58121-9b17-4569-8d79-6fa63785d921 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:36.032598 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:36.038166 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 571/1101] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:26:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 31 14:26:36.043814 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552b36f-fdce-4484-aeb0-87c8b76b2b61 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:36.108080 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b58121-9b17-4569-8d79-6fa63785d921 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:36.142704 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf7dee-b018-4e77-87e2-5253a05d48c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:36.142937 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552b36f-fdce-4484-aeb0-87c8b76b2b61 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:36.143504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf7dee-b018-4e77-87e2-5253a05d48c9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:36.143787 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf7dee-b018-4e77-87e2-5253a05d48c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:36.144059 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf7dee-b018-4e77-87e2-5253a05d48c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:36.150620 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf7dee-b018-4e77-87e2-5253a05d48c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:36.152075 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf7dee-b018-4e77-87e2-5253a05d48c9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:36.165977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-adaf7dee-b018-4e77-87e2-5253a05d48c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:26:36.165977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:36.165977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:36.165977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:36.165977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:36.165977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:36.165977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:36.165977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:36.165977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:36.165977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:36.165977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:36.165977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:36.165977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:26:36.165977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:26:36.165977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:26:36.166955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:26:36.166955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:36.166955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:36.166955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:36.166955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:36.166955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:36.166955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:36.166955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:36.166955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:26:36.166955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:36.166955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:36.166955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:36.166955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:36.166955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:36.166955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:36.166955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:36.166955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:36.167873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:26:36.167873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:26:36.167873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:26:36.167873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:26:36.167873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:26:36.167873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:26:36.167873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:26:36.167873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:26:36.167873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 531/1102] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:26:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:26:36.172860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf7dee-b018-4e77-87e2-5253a05d48c9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:36.175905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959be262-5823-4d8a-9966-797d4c4fcdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:36.176756 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959be262-5823-4d8a-9966-797d4c4fcdc0 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:36.177032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959be262-5823-4d8a-9966-797d4c4fcdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:36.177303 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959be262-5823-4d8a-9966-797d4c4fcdc0 None admin] PATH_INFO: `/v3/projects/b72b6db8b7ee4acdbe588f86788a5c3e` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:36.178461 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959be262-5823-4d8a-9966-797d4c4fcdc0 None admin] RBAC: Authorizing `identity:get_project(project_id=b72b6db8b7ee4acdbe588f86788a5c3e)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:36.179931 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959be262-5823-4d8a-9966-797d4c4fcdc0 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:36.186973 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 572/1103] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:26:36 2022] GET /identity/v3/projects/b72b6db8b7ee4acdbe588f86788a5c3e => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:36.188135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959be262-5823-4d8a-9966-797d4c4fcdc0 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:36.228877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:36.253410 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:36.272205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf7dee-b018-4e77-87e2-5253a05d48c9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:36.286924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959be262-5823-4d8a-9966-797d4c4fcdc0 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:36.306931 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412cbbe3-eb36-47bb-b39c-5009f6daf09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:36.307596 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412cbbe3-eb36-47bb-b39c-5009f6daf09d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:36.307883 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412cbbe3-eb36-47bb-b39c-5009f6daf09d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:36.308160 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412cbbe3-eb36-47bb-b39c-5009f6daf09d None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:36.308946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412cbbe3-eb36-47bb-b39c-5009f6daf09d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:36.309260 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412cbbe3-eb36-47bb-b39c-5009f6daf09d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:36.310704 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412cbbe3-eb36-47bb-b39c-5009f6daf09d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:36.322154 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e4c98-440d-476e-a299-7e276a8334ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:36.322399 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 532/1104] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:26:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:36.327559 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e4c98-440d-476e-a299-7e276a8334ed None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:36.327698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e4c98-440d-476e-a299-7e276a8334ed None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:36.327935 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e4c98-440d-476e-a299-7e276a8334ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:36.328345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412cbbe3-eb36-47bb-b39c-5009f6daf09d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:36.328801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e4c98-440d-476e-a299-7e276a8334ed None admin] RBAC: Adding query filter params (role.id=e8d679596ffb45939229d1e2a13ce9b0, scope.project.id=b72b6db8b7ee4acdbe588f86788a5c3e, user.id=5fbe12d725eb40879e0e09a3a1439b7a) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:36.329120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e4c98-440d-476e-a299-7e276a8334ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:36.330534 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e4c98-440d-476e-a299-7e276a8334ed None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:36.336521 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 573/1105] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:26:36 2022] GET /identity/v3/role_assignments?user.id=5fbe12d725eb40879e0e09a3a1439b7a&scope.project.id=b72b6db8b7ee4acdbe588f86788a5c3e&role.id=e8d679596ffb45939229d1e2a13ce9b0 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:36.434456 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412cbbe3-eb36-47bb-b39c-5009f6daf09d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:36.451936 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:36.470595 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deadb2b0-faad-413c-9881-e852ec1c46d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:36.471296 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadb2b0-faad-413c-9881-e852ec1c46d5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:36.471583 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadb2b0-faad-413c-9881-e852ec1c46d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:36.471862 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadb2b0-faad-413c-9881-e852ec1c46d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:36.472732 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deadb2b0-faad-413c-9881-e852ec1c46d5 None admin] RBAC: Adding query filter params (role.id=8de91fffa27842109896bf18045db796, scope.project.id=f965a93a2f674598890bd8cafededb2c, user.id=68c6363b60954c3ebf50d2019f5f2fd2) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:36.473046 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deadb2b0-faad-413c-9881-e852ec1c46d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:36.474488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deadb2b0-faad-413c-9881-e852ec1c46d5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:36.481830 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 533/1106] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:26:36 2022] GET /identity/v3/role_assignments?user.id=68c6363b60954c3ebf50d2019f5f2fd2&scope.project.id=f965a93a2f674598890bd8cafededb2c&role.id=8de91fffa27842109896bf18045db796 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:38.635138 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405008c3-86dd-433c-9da9-fbe1eb2222d5 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:38.635638 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405008c3-86dd-433c-9da9-fbe1eb2222d5 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:38.635752 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405008c3-86dd-433c-9da9-fbe1eb2222d5 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:38.637065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 574/1107] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:38.642355 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347e117-54d8-4090-8305-a199b08a2bbb None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:38.642802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347e117-54d8-4090-8305-a199b08a2bbb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:38.642933 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347e117-54d8-4090-8305-a199b08a2bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:38.659813 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c347e117-54d8-4090-8305-a199b08a2bbb None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:38.660982 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c347e117-54d8-4090-8305-a199b08a2bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:38.724329 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 534/1108] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:38.748717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405008c3-86dd-433c-9da9-fbe1eb2222d5 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:38.847603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405008c3-86dd-433c-9da9-fbe1eb2222d5 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:38.874970 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:38.880841 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ede13c-3e7d-4ab6-82ab-6efef0e528f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:38.881617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ede13c-3e7d-4ab6-82ab-6efef0e528f2 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:38.881897 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ede13c-3e7d-4ab6-82ab-6efef0e528f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:38.882170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ede13c-3e7d-4ab6-82ab-6efef0e528f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:38.883323 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ede13c-3e7d-4ab6-82ab-6efef0e528f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:38.885344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ede13c-3e7d-4ab6-82ab-6efef0e528f2 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:38.886927 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 575/1109] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:26:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:38.892477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347e117-54d8-4090-8305-a199b08a2bbb None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:38.988957 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347e117-54d8-4090-8305-a199b08a2bbb None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:39.000232 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:39.023526 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aff8e0-28d9-4b8d-8720-2fde1d795056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:39.024200 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aff8e0-28d9-4b8d-8720-2fde1d795056 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:39.024514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aff8e0-28d9-4b8d-8720-2fde1d795056 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:39.024836 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aff8e0-28d9-4b8d-8720-2fde1d795056 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:39.025554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aff8e0-28d9-4b8d-8720-2fde1d795056 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:39.027011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aff8e0-28d9-4b8d-8720-2fde1d795056 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:39.045612 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 535/1110] 10.222.0.67 () {60 vars in 1132 bytes} [Tue May 31 14:26:38 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:26:41.292245 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3e260-94c1-4fea-802f-ac085e386187 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:41.292863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3e260-94c1-4fea-802f-ac085e386187 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:41.292863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3e260-94c1-4fea-802f-ac085e386187 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:41.294161 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 576/1111] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:41.299255 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56558c09-fd05-4c5d-b413-591460167b1b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:41.299686 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56558c09-fd05-4c5d-b413-591460167b1b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:41.299819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56558c09-fd05-4c5d-b413-591460167b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:41.316138 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-56558c09-fd05-4c5d-b413-591460167b1b None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:41.317334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-56558c09-fd05-4c5d-b413-591460167b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:41.381610 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 536/1112] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:41.404909 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3e260-94c1-4fea-802f-ac085e386187 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:41.502474 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3e260-94c1-4fea-802f-ac085e386187 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:41.535601 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8976d-47cb-4c70-bdbb-078653fcf66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:41.536439 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8976d-47cb-4c70-bdbb-078653fcf66d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:41.536722 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8976d-47cb-4c70-bdbb-078653fcf66d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:41.536997 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8976d-47cb-4c70-bdbb-078653fcf66d None admin] PATH_INFO: `/v3/roles/1b9c18d4b8b540bf9825080748cec3bc` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:41.538445 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8976d-47cb-4c70-bdbb-078653fcf66d None admin] RBAC: Authorizing `identity:get_role(role_id=1b9c18d4b8b540bf9825080748cec3bc)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:41.539870 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8976d-47cb-4c70-bdbb-078653fcf66d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:41.541205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 577/1113] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:26:41 2022] GET /identity/v3/roles/1b9c18d4b8b540bf9825080748cec3bc => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:41.546787 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56558c09-fd05-4c5d-b413-591460167b1b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:41.635781 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:41.642203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56558c09-fd05-4c5d-b413-591460167b1b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:41.674942 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab049c9a-ec84-4d47-bd32-4f544e89f591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:41.675804 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab049c9a-ec84-4d47-bd32-4f544e89f591 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:41.676088 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab049c9a-ec84-4d47-bd32-4f544e89f591 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:41.676365 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab049c9a-ec84-4d47-bd32-4f544e89f591 None admin] PATH_INFO: `/v3/users/02a5b61e9b5c40ffae8edd01bad7931d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:41.688953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab049c9a-ec84-4d47-bd32-4f544e89f591 None admin] RBAC: Authorizing `identity:get_user(user_id=02a5b61e9b5c40ffae8edd01bad7931d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:41.690372 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab049c9a-ec84-4d47-bd32-4f544e89f591 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:41.692020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 537/1114] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:26:41 2022] GET /identity/v3/users/02a5b61e9b5c40ffae8edd01bad7931d => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:41.698082 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8976d-47cb-4c70-bdbb-078653fcf66d None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:41.760734 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:41.794085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8976d-47cb-4c70-bdbb-078653fcf66d None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:41.826738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf93b0-31ae-4bd4-ae9c-85d7232b1791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:41.827364 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf93b0-31ae-4bd4-ae9c-85d7232b1791 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:41.827641 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf93b0-31ae-4bd4-ae9c-85d7232b1791 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:41.827912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf93b0-31ae-4bd4-ae9c-85d7232b1791 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:41.891887 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf93b0-31ae-4bd4-ae9c-85d7232b1791 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:41.893260 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf93b0-31ae-4bd4-ae9c-85d7232b1791 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:41.955689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:41.989946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 578/1115] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:26:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 31 14:26:41.996375 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab049c9a-ec84-4d47-bd32-4f544e89f591 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:42.092508 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab049c9a-ec84-4d47-bd32-4f544e89f591 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:42.132524 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf6522-aee3-4b57-94aa-e48a29aa0ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:42.134511 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf6522-aee3-4b57-94aa-e48a29aa0ddd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:42.134996 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf6522-aee3-4b57-94aa-e48a29aa0ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:42.135431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf6522-aee3-4b57-94aa-e48a29aa0ddd None admin] PATH_INFO: `/v3/projects/b72b6db8b7ee4acdbe588f86788a5c3e` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:42.136914 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf6522-aee3-4b57-94aa-e48a29aa0ddd None admin] RBAC: Authorizing `identity:get_project(project_id=b72b6db8b7ee4acdbe588f86788a5c3e)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:42.138339 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf6522-aee3-4b57-94aa-e48a29aa0ddd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:42.140080 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 538/1116] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:26:41 2022] GET /identity/v3/projects/b72b6db8b7ee4acdbe588f86788a5c3e => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:42.146286 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf93b0-31ae-4bd4-ae9c-85d7232b1791 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:42.240777 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf93b0-31ae-4bd4-ae9c-85d7232b1791 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:42.273003 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee34c7-70ed-453e-a5eb-2cd504e9650c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:42.273639 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee34c7-70ed-453e-a5eb-2cd504e9650c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:42.273916 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee34c7-70ed-453e-a5eb-2cd504e9650c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:42.274211 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee34c7-70ed-453e-a5eb-2cd504e9650c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:42.275055 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee34c7-70ed-453e-a5eb-2cd504e9650c None admin] RBAC: Adding query filter params (role.id=1b9c18d4b8b540bf9825080748cec3bc, scope.project.id=b72b6db8b7ee4acdbe588f86788a5c3e, user.id=02a5b61e9b5c40ffae8edd01bad7931d) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:42.275367 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee34c7-70ed-453e-a5eb-2cd504e9650c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:42.276793 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee34c7-70ed-453e-a5eb-2cd504e9650c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:42.281989 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 579/1117] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:26:42 2022] GET /identity/v3/role_assignments?user.id=02a5b61e9b5c40ffae8edd01bad7931d&scope.project.id=b72b6db8b7ee4acdbe588f86788a5c3e&role.id=1b9c18d4b8b540bf9825080748cec3bc => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:44.501303 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fa2cc-87d3-43a8-afd6-1d506c511070 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:44.501799 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fa2cc-87d3-43a8-afd6-1d506c511070 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:44.501799 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fa2cc-87d3-43a8-afd6-1d506c511070 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:44.503213 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 539/1118] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:44.508315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10d989-bb4a-4837-8d14-413314326757 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:44.509208 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10d989-bb4a-4837-8d14-413314326757 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:44.509404 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10d989-bb4a-4837-8d14-413314326757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:44.526273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-ea10d989-bb4a-4837-8d14-413314326757 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:44.527489 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ea10d989-bb4a-4837-8d14-413314326757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:44.549933 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:44.592311 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 580/1119] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:44.614012 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fa2cc-87d3-43a8-afd6-1d506c511070 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:44.637929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:44.713087 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fa2cc-87d3-43a8-afd6-1d506c511070 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:44.744860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb42c4e-e94f-468a-a994-d9db9ef8bec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:44.745706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb42c4e-e94f-468a-a994-d9db9ef8bec4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:44.745990 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb42c4e-e94f-468a-a994-d9db9ef8bec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:44.746266 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb42c4e-e94f-468a-a994-d9db9ef8bec4 None admin] PATH_INFO: `/v3/roles/1b9c18d4b8b540bf9825080748cec3bc` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:44.747748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb42c4e-e94f-468a-a994-d9db9ef8bec4 None admin] RBAC: Authorizing `identity:get_role(role_id=1b9c18d4b8b540bf9825080748cec3bc)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:44.749108 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb42c4e-e94f-468a-a994-d9db9ef8bec4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:44.750450 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 540/1120] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:26:44 2022] GET /identity/v3/roles/1b9c18d4b8b540bf9825080748cec3bc => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:44.756103 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10d989-bb4a-4837-8d14-413314326757 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:44.852321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10d989-bb4a-4837-8d14-413314326757 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:44.885439 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881e4d2-8265-4569-a870-420b05a2bf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:44.886296 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881e4d2-8265-4569-a870-420b05a2bf59 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:44.886579 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881e4d2-8265-4569-a870-420b05a2bf59 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:44.886883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881e4d2-8265-4569-a870-420b05a2bf59 None admin] PATH_INFO: `/v3/users/02a5b61e9b5c40ffae8edd01bad7931d` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:44.888058 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881e4d2-8265-4569-a870-420b05a2bf59 None admin] RBAC: Authorizing `identity:get_user(user_id=02a5b61e9b5c40ffae8edd01bad7931d)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:44.889490 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881e4d2-8265-4569-a870-420b05a2bf59 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:44.891130 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 581/1121] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:26:44 2022] GET /identity/v3/users/02a5b61e9b5c40ffae8edd01bad7931d => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:44.896516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb42c4e-e94f-468a-a994-d9db9ef8bec4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:44.983739 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:44.994027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb42c4e-e94f-468a-a994-d9db9ef8bec4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:45.026777 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4d78a-1180-4c46-aa38-3ec388470e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:45.027577 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4d78a-1180-4c46-aa38-3ec388470e4e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:45.027865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4d78a-1180-4c46-aa38-3ec388470e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:45.028146 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4d78a-1180-4c46-aa38-3ec388470e4e None admin] PATH_INFO: `/v3/projects/b72b6db8b7ee4acdbe588f86788a5c3e` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:45.029315 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4d78a-1180-4c46-aa38-3ec388470e4e None admin] RBAC: Authorizing `identity:get_project(project_id=b72b6db8b7ee4acdbe588f86788a5c3e)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:45.030771 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4d78a-1180-4c46-aa38-3ec388470e4e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:45.032440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 541/1122] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:26:44 2022] GET /identity/v3/projects/b72b6db8b7ee4acdbe588f86788a5c3e => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:26:45.038276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881e4d2-8265-4569-a870-420b05a2bf59 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:45.066697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:45.136704 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881e4d2-8265-4569-a870-420b05a2bf59 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:45.169419 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001998f-0da9-4aac-9a6a-0f22f0e012a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:45.170296 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001998f-0da9-4aac-9a6a-0f22f0e012a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:45.170582 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001998f-0da9-4aac-9a6a-0f22f0e012a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:45.170886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001998f-0da9-4aac-9a6a-0f22f0e012a9 None admin] PATH_INFO: `/v3/projects/b72b6db8b7ee4acdbe588f86788a5c3e/users/02a5b61e9b5c40ffae8edd01bad7931d/roles/1b9c18d4b8b540bf9825080748cec3bc` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:45.173081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001998f-0da9-4aac-9a6a-0f22f0e012a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=b72b6db8b7ee4acdbe588f86788a5c3e, user_id=02a5b61e9b5c40ffae8edd01bad7931d, role_id=1b9c18d4b8b540bf9825080748cec3bc)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:45.174507 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001998f-0da9-4aac-9a6a-0f22f0e012a9 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:45.187707 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 582/1123] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:26:45 2022] PUT /identity/v3/projects/b72b6db8b7ee4acdbe588f86788a5c3e/users/02a5b61e9b5c40ffae8edd01bad7931d/roles/1b9c18d4b8b540bf9825080748cec3bc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:26:47.391262 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd4db3-3d10-4030-bd2b-9f8bac387258 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:47.391760 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd4db3-3d10-4030-bd2b-9f8bac387258 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:47.391760 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd4db3-3d10-4030-bd2b-9f8bac387258 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:47.393109 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 542/1124] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:47.398448 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c18f9-a0e2-4644-b89d-fecad36dca05 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:47.398897 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c18f9-a0e2-4644-b89d-fecad36dca05 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:47.399559 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c18f9-a0e2-4644-b89d-fecad36dca05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:47.418222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-0d4c18f9-a0e2-4644-b89d-fecad36dca05 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:47.419462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0d4c18f9-a0e2-4644-b89d-fecad36dca05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:47.484411 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 583/1125] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:47.508150 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd4db3-3d10-4030-bd2b-9f8bac387258 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:47.606229 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd4db3-3d10-4030-bd2b-9f8bac387258 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:47.638726 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4eeb3-09e3-4a07-8b1e-06ccde5ba9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:47.639566 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4eeb3-09e3-4a07-8b1e-06ccde5ba9db None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:47.639849 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4eeb3-09e3-4a07-8b1e-06ccde5ba9db None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:47.640123 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4eeb3-09e3-4a07-8b1e-06ccde5ba9db None admin] PATH_INFO: `/v3/roles/1b9c18d4b8b540bf9825080748cec3bc` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:47.641615 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4eeb3-09e3-4a07-8b1e-06ccde5ba9db None admin] RBAC: Authorizing `identity:get_role(role_id=1b9c18d4b8b540bf9825080748cec3bc)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:47.642937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4eeb3-09e3-4a07-8b1e-06ccde5ba9db None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:47.644263 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 543/1126] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:26:47 2022] GET /identity/v3/roles/1b9c18d4b8b540bf9825080748cec3bc => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:47.650549 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c18f9-a0e2-4644-b89d-fecad36dca05 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:47.668132 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:47.746122 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c18f9-a0e2-4644-b89d-fecad36dca05 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:47.779538 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ffa79a-0c61-4620-98b8-0447af3c1297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:47.780403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ffa79a-0c61-4620-98b8-0447af3c1297 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:47.780687 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ffa79a-0c61-4620-98b8-0447af3c1297 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:47.780960 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ffa79a-0c61-4620-98b8-0447af3c1297 None admin] PATH_INFO: `/v3/users/02a5b61e9b5c40ffae8edd01bad7931d` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:47.782141 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ffa79a-0c61-4620-98b8-0447af3c1297 None admin] RBAC: Authorizing `identity:get_user(user_id=02a5b61e9b5c40ffae8edd01bad7931d)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:47.783603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ffa79a-0c61-4620-98b8-0447af3c1297 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:47.785210 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 584/1127] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:26:47 2022] GET /identity/v3/users/02a5b61e9b5c40ffae8edd01bad7931d => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:47.790984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4eeb3-09e3-4a07-8b1e-06ccde5ba9db None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:47.816555 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:47.887479 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4eeb3-09e3-4a07-8b1e-06ccde5ba9db None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:47.919725 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b29b4e-a71e-4a2c-b056-f714f909f9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:47.920311 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b29b4e-a71e-4a2c-b056-f714f909f9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:47.920599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b29b4e-a71e-4a2c-b056-f714f909f9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:47.920880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b29b4e-a71e-4a2c-b056-f714f909f9e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:47.985475 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b29b4e-a71e-4a2c-b056-f714f909f9e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:47.986922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b29b4e-a71e-4a2c-b056-f714f909f9e9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:48.012527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:48.082620 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 544/1128] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:26:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 31 14:26:48.088355 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ffa79a-0c61-4620-98b8-0447af3c1297 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:48.170917 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:48.185940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ffa79a-0c61-4620-98b8-0447af3c1297 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:48.218471 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af070a2-b229-47e6-a4d6-5ba839115e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:48.219346 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af070a2-b229-47e6-a4d6-5ba839115e5a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:48.219627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af070a2-b229-47e6-a4d6-5ba839115e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:48.219899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af070a2-b229-47e6-a4d6-5ba839115e5a None admin] PATH_INFO: `/v3/projects/b72b6db8b7ee4acdbe588f86788a5c3e` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:48.221061 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af070a2-b229-47e6-a4d6-5ba839115e5a None admin] RBAC: Authorizing `identity:get_project(project_id=b72b6db8b7ee4acdbe588f86788a5c3e)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:48.222816 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af070a2-b229-47e6-a4d6-5ba839115e5a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:48.224498 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 585/1129] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:26:48 2022] GET /identity/v3/projects/b72b6db8b7ee4acdbe588f86788a5c3e => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:48.231086 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b29b4e-a71e-4a2c-b056-f714f909f9e9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:48.326870 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b29b4e-a71e-4a2c-b056-f714f909f9e9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:48.351132 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:48.360763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6fc105-95ae-43e7-98bc-b8bf6293ed92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:48.361413 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fc105-95ae-43e7-98bc-b8bf6293ed92 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:48.361702 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fc105-95ae-43e7-98bc-b8bf6293ed92 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:48.361980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fc105-95ae-43e7-98bc-b8bf6293ed92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:48.362844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6fc105-95ae-43e7-98bc-b8bf6293ed92 None admin] RBAC: Adding query filter params (role.id=1b9c18d4b8b540bf9825080748cec3bc, scope.project.id=b72b6db8b7ee4acdbe588f86788a5c3e, user.id=02a5b61e9b5c40ffae8edd01bad7931d) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:48.363162 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6fc105-95ae-43e7-98bc-b8bf6293ed92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:48.364578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6fc105-95ae-43e7-98bc-b8bf6293ed92 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:48.370987 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 545/1130] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:26:48 2022] GET /identity/v3/role_assignments?user.id=02a5b61e9b5c40ffae8edd01bad7931d&scope.project.id=b72b6db8b7ee4acdbe588f86788a5c3e&role.id=1b9c18d4b8b540bf9825080748cec3bc => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:50.619295 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6b80b-8d84-43bc-b0dc-a1da3328ef82 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:50.619821 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6b80b-8d84-43bc-b0dc-a1da3328ef82 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:50.619932 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6b80b-8d84-43bc-b0dc-a1da3328ef82 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:50.621231 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 586/1131] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:50.626566 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41cab10-dd95-4778-a642-309846d17319 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:50.627008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41cab10-dd95-4778-a642-309846d17319 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:50.627161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41cab10-dd95-4778-a642-309846d17319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:50.643577 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b41cab10-dd95-4778-a642-309846d17319 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:50.644784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b41cab10-dd95-4778-a642-309846d17319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:50.708775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 546/1132] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:50.732203 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6b80b-8d84-43bc-b0dc-a1da3328ef82 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:50.830527 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6b80b-8d84-43bc-b0dc-a1da3328ef82 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:50.864451 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61164386-9a81-41ab-aeba-0a13bcd1d361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:50.865226 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61164386-9a81-41ab-aeba-0a13bcd1d361 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:50.865508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61164386-9a81-41ab-aeba-0a13bcd1d361 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:50.865782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61164386-9a81-41ab-aeba-0a13bcd1d361 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:50.866975 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61164386-9a81-41ab-aeba-0a13bcd1d361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:50.868491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61164386-9a81-41ab-aeba-0a13bcd1d361 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:50.870145 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 587/1133] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:26:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:50.875678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41cab10-dd95-4778-a642-309846d17319 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:50.971468 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41cab10-dd95-4778-a642-309846d17319 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:50.991697 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:51.005429 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fda32-06eb-46f3-92dc-f0e5ee859675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:51.006086 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fda32-06eb-46f3-92dc-f0e5ee859675 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:51.006374 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fda32-06eb-46f3-92dc-f0e5ee859675 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:51.006669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fda32-06eb-46f3-92dc-f0e5ee859675 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:51.007390 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fda32-06eb-46f3-92dc-f0e5ee859675 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:51.008857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fda32-06eb-46f3-92dc-f0e5ee859675 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:51.033963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 547/1134] 10.222.0.67 () {60 vars in 1141 bytes} [Tue May 31 14:26:50 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:26:53.264068 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c6caf-0622-4216-a5c3-081931726693 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:53.264590 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c6caf-0622-4216-a5c3-081931726693 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:53.264710 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c6caf-0622-4216-a5c3-081931726693 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:53.265992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 588/1135] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:53.271136 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc3986-be55-403c-a53f-af4a55cd4736 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:53.271512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc3986-be55-403c-a53f-af4a55cd4736 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:53.271698 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc3986-be55-403c-a53f-af4a55cd4736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:53.288231 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-fefc3986-be55-403c-a53f-af4a55cd4736 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:53.289444 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fefc3986-be55-403c-a53f-af4a55cd4736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:53.354124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 548/1136] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 31 14:26:53.378054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c6caf-0622-4216-a5c3-081931726693 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:53.402330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:53.479226 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c6caf-0622-4216-a5c3-081931726693 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:53.511557 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48e3f2-3058-460e-8f1b-d335ec9bbe37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:53.512341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48e3f2-3058-460e-8f1b-d335ec9bbe37 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:53.512627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48e3f2-3058-460e-8f1b-d335ec9bbe37 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:53.512905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48e3f2-3058-460e-8f1b-d335ec9bbe37 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:53.514059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48e3f2-3058-460e-8f1b-d335ec9bbe37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:53.515510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48e3f2-3058-460e-8f1b-d335ec9bbe37 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:53.517123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 589/1137] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:26:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:53.522875 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc3986-be55-403c-a53f-af4a55cd4736 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:53.618773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc3986-be55-403c-a53f-af4a55cd4736 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:53.620236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:53.652677 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4bdb28-8075-458b-b08e-98e51afb2fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:53.653286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4bdb28-8075-458b-b08e-98e51afb2fcc None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:53.653573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4bdb28-8075-458b-b08e-98e51afb2fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:53.653847 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4bdb28-8075-458b-b08e-98e51afb2fcc None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:53.654547 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4bdb28-8075-458b-b08e-98e51afb2fcc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:53.656005 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4bdb28-8075-458b-b08e-98e51afb2fcc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:53.674784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 549/1138] 10.222.0.67 () {60 vars in 1132 bytes} [Tue May 31 14:26:53 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:26:55.925206 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cd7f7-d332-44dc-ada7-fd696ebef797 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:55.925761 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cd7f7-d332-44dc-ada7-fd696ebef797 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:55.925761 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cd7f7-d332-44dc-ada7-fd696ebef797 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:55.927182 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 590/1139] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:55.933083 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44cc14-2877-47cc-8f78-cd10a0119ff4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:55.933500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44cc14-2877-47cc-8f78-cd10a0119ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:55.933674 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44cc14-2877-47cc-8f78-cd10a0119ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:55.950171 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1d44cc14-2877-47cc-8f78-cd10a0119ff4 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:55.951372 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1d44cc14-2877-47cc-8f78-cd10a0119ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:56.014558 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 550/1140] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:56.037665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cd7f7-d332-44dc-ada7-fd696ebef797 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:56.128538 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:56.140346 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cd7f7-d332-44dc-ada7-fd696ebef797 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:56.175085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad6b7e-87fd-4aa2-b3c9-e4dd5ad7c5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:56.175919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad6b7e-87fd-4aa2-b3c9-e4dd5ad7c5da None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:56.176145 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad6b7e-87fd-4aa2-b3c9-e4dd5ad7c5da None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:56.176418 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad6b7e-87fd-4aa2-b3c9-e4dd5ad7c5da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:56.186166 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad6b7e-87fd-4aa2-b3c9-e4dd5ad7c5da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:26:56.193651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-63ad6b7e-87fd-4aa2-b3c9-e4dd5ad7c5da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:26:56.193651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:56.193651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:56.193651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:56.193651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:56.193651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:56.193651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:56.193651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:56.193651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:56.193651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:56.193651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:56.193651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:56.193651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:26:56.193651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:26:56.193651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:26:56.193651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:26:56.194783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:26:56.194783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:26:56.194783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:26:56.194783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:26:56.194783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:56.194783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:56.194783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:56.194783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:56.194783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:56.194783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:56.194783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:56.194783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:26:56.194783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:56.194783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:56.194783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:56.194783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:56.195839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:56.195839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:56.195839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:56.195839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:56.195839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:26:56.195839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:26:56.195839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:26:56.195839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:26:56.195839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:26:56.195839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:26:56.195839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:26:56.195839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:56.195839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 591/1141] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:26:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:56.201098 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44cc14-2877-47cc-8f78-cd10a0119ff4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:56.284114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:56.297602 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44cc14-2877-47cc-8f78-cd10a0119ff4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:56.330452 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87761e-b224-4773-ba3b-965fdb9ab8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:56.331326 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87761e-b224-4773-ba3b-965fdb9ab8ff None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:56.331616 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87761e-b224-4773-ba3b-965fdb9ab8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:56.331894 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87761e-b224-4773-ba3b-965fdb9ab8ff None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:56.332639 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f87761e-b224-4773-ba3b-965fdb9ab8ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:56.332948 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f87761e-b224-4773-ba3b-965fdb9ab8ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:56.334307 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f87761e-b224-4773-ba3b-965fdb9ab8ff None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:56.342910 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 551/1142] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:26:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:56.348315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad6b7e-87fd-4aa2-b3c9-e4dd5ad7c5da None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:56.444691 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad6b7e-87fd-4aa2-b3c9-e4dd5ad7c5da None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:56.478055 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c49a48-3bed-4168-8c43-6de6ccc0e5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:56.478958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c49a48-3bed-4168-8c43-6de6ccc0e5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:56.479249 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c49a48-3bed-4168-8c43-6de6ccc0e5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:56.479522 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c49a48-3bed-4168-8c43-6de6ccc0e5f7 None admin] PATH_INFO: `/v3/users/2d78b255174e4b84ab91333d261de646` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:56.492040 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c49a48-3bed-4168-8c43-6de6ccc0e5f7 None admin] RBAC: Authorizing `identity:get_user(user_id=2d78b255174e4b84ab91333d261de646)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:56.493538 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c49a48-3bed-4168-8c43-6de6ccc0e5f7 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:56.495210 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 592/1143] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:26:56 2022] GET /identity/v3/users/2d78b255174e4b84ab91333d261de646 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:56.501133 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87761e-b224-4773-ba3b-965fdb9ab8ff None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:56.597914 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87761e-b224-4773-ba3b-965fdb9ab8ff None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:56.629603 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bd3a5-15e5-4bc9-805c-a85814913c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:56.630244 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bd3a5-15e5-4bc9-805c-a85814913c31 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:56.630531 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bd3a5-15e5-4bc9-805c-a85814913c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:56.630844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bd3a5-15e5-4bc9-805c-a85814913c31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:56.649421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:56.696611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3bd3a5-15e5-4bc9-805c-a85814913c31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:56.697993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3bd3a5-15e5-4bc9-805c-a85814913c31 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:56.792861 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 552/1144] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:26:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 31 14:26:56.798918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c49a48-3bed-4168-8c43-6de6ccc0e5f7 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:56.809294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:56.895063 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c49a48-3bed-4168-8c43-6de6ccc0e5f7 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:56.928808 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b94f6f-a2cd-476d-a9ec-08a70089463d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:56.929680 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b94f6f-a2cd-476d-a9ec-08a70089463d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:56.930443 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b94f6f-a2cd-476d-a9ec-08a70089463d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:56.930739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b94f6f-a2cd-476d-a9ec-08a70089463d None admin] PATH_INFO: `/v3/projects/9b2fa8031dc2446687372008e202c827` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:56.931890 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b94f6f-a2cd-476d-a9ec-08a70089463d None admin] RBAC: Authorizing `identity:get_project(project_id=9b2fa8031dc2446687372008e202c827)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:56.933309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b94f6f-a2cd-476d-a9ec-08a70089463d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:56.935005 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 593/1145] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:26:56 2022] GET /identity/v3/projects/9b2fa8031dc2446687372008e202c827 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:56.940985 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bd3a5-15e5-4bc9-805c-a85814913c31 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:56.988680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:57.036513 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bd3a5-15e5-4bc9-805c-a85814913c31 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:57.068566 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cace8f6-ba74-4d61-9d51-d07eb9b207be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:57.069202 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cace8f6-ba74-4d61-9d51-d07eb9b207be None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:57.069485 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cace8f6-ba74-4d61-9d51-d07eb9b207be None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:57.069760 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cace8f6-ba74-4d61-9d51-d07eb9b207be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:57.071167 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cace8f6-ba74-4d61-9d51-d07eb9b207be None admin] RBAC: Adding query filter params (role.id=e8d679596ffb45939229d1e2a13ce9b0, scope.project.id=9b2fa8031dc2446687372008e202c827, user.id=2d78b255174e4b84ab91333d261de646) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:57.071480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cace8f6-ba74-4d61-9d51-d07eb9b207be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:57.072888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cace8f6-ba74-4d61-9d51-d07eb9b207be None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:57.078385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 553/1146] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:26:56 2022] GET /identity/v3/role_assignments?user.id=2d78b255174e4b84ab91333d261de646&scope.project.id=9b2fa8031dc2446687372008e202c827&role.id=e8d679596ffb45939229d1e2a13ce9b0 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:59.257010 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd556b-a938-4092-98e5-568aa627afce None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:59.257513 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd556b-a938-4092-98e5-568aa627afce None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:59.257605 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd556b-a938-4092-98e5-568aa627afce None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:59.258981 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 594/1147] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:26:59.264109 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fe5ca-3443-4789-9c7d-9e650d0a00bd None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:59.264486 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fe5ca-3443-4789-9c7d-9e650d0a00bd None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:59.264677 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fe5ca-3443-4789-9c7d-9e650d0a00bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:59.281538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-dd3fe5ca-3443-4789-9c7d-9e650d0a00bd None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:26:59.282758 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dd3fe5ca-3443-4789-9c7d-9e650d0a00bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:26:59.347024 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 554/1148] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:26:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:26:59.369472 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd556b-a938-4092-98e5-568aa627afce None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:59.443242 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:59.467933 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd556b-a938-4092-98e5-568aa627afce None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:59.500260 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d7131-b717-474a-9566-89dc03100c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:59.501040 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d7131-b717-474a-9566-89dc03100c52 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:59.501320 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d7131-b717-474a-9566-89dc03100c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:59.501592 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d7131-b717-474a-9566-89dc03100c52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:59.511801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d7131-b717-474a-9566-89dc03100c52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:26:59.519235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-738d7131-b717-474a-9566-89dc03100c52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:26:59.519235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:26:59.519235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:26:59.519235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:26:59.519235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:26:59.519235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:26:59.519235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:26:59.519235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:26:59.519235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:26:59.519235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:26:59.519235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:26:59.519235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:26:59.519235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:26:59.519235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:26:59.519235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:26:59.519235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:26:59.520306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:26:59.520306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:26:59.520306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:26:59.520306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:26:59.520306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:26:59.520306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:26:59.520306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:26:59.520306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:26:59.520306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:26:59.520306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:26:59.520306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:26:59.520306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:26:59.520306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:26:59.520306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:26:59.520306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:26:59.520306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:26:59.521263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:26:59.521263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:26:59.521263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:26:59.521263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:26:59.521263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:26:59.521263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:26:59.521263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:26:59.521263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:26:59.521263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:26:59.521263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:26:59.521263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:26:59.521263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:26:59.521263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 595/1149] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:26:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:26:59.526949 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3fe5ca-3443-4789-9c7d-9e650d0a00bd None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:59.569179 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:59.621032 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3fe5ca-3443-4789-9c7d-9e650d0a00bd None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:59.653711 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3185ed-77b3-425e-9415-ff2ae81233ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:59.654367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3185ed-77b3-425e-9415-ff2ae81233ff None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:59.654680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3185ed-77b3-425e-9415-ff2ae81233ff None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:59.654965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3185ed-77b3-425e-9415-ff2ae81233ff None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:59.655730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3185ed-77b3-425e-9415-ff2ae81233ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:26:59.656043 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3185ed-77b3-425e-9415-ff2ae81233ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:59.657394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3185ed-77b3-425e-9415-ff2ae81233ff None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:59.665155 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 555/1150] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:26:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:59.670877 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d7131-b717-474a-9566-89dc03100c52 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:59.766226 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d7131-b717-474a-9566-89dc03100c52 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:59.798172 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596aeb82-f580-41eb-92a1-2d7cbaaecb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:59.799055 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596aeb82-f580-41eb-92a1-2d7cbaaecb20 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:59.799332 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596aeb82-f580-41eb-92a1-2d7cbaaecb20 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:59.799606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596aeb82-f580-41eb-92a1-2d7cbaaecb20 None admin] PATH_INFO: `/v3/users/2d78b255174e4b84ab91333d261de646` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:59.800766 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596aeb82-f580-41eb-92a1-2d7cbaaecb20 None admin] RBAC: Authorizing `identity:get_user(user_id=2d78b255174e4b84ab91333d261de646)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:59.802176 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596aeb82-f580-41eb-92a1-2d7cbaaecb20 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:59.803788 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 596/1151] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:26:59 2022] GET /identity/v3/users/2d78b255174e4b84ab91333d261de646 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:59.809305 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3185ed-77b3-425e-9415-ff2ae81233ff None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:59.904935 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3185ed-77b3-425e-9415-ff2ae81233ff None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:26:59.911464 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:26:59.937648 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65f94c-a2d3-4dc0-8b2b-d9465db62274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:26:59.938498 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65f94c-a2d3-4dc0-8b2b-d9465db62274 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:26:59.938811 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65f94c-a2d3-4dc0-8b2b-d9465db62274 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:26:59.939092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65f94c-a2d3-4dc0-8b2b-d9465db62274 None admin] PATH_INFO: `/v3/projects/9b2fa8031dc2446687372008e202c827` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:26:59.940257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65f94c-a2d3-4dc0-8b2b-d9465db62274 None admin] RBAC: Authorizing `identity:get_project(project_id=9b2fa8031dc2446687372008e202c827)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:26:59.941668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65f94c-a2d3-4dc0-8b2b-d9465db62274 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:26:59.943368 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 556/1152] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:26:59 2022] GET /identity/v3/projects/9b2fa8031dc2446687372008e202c827 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:26:59.949937 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596aeb82-f580-41eb-92a1-2d7cbaaecb20 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:26:59.953568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:00.043851 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596aeb82-f580-41eb-92a1-2d7cbaaecb20 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:00.076185 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df858b4-efa3-433c-9dd0-00a502a3a3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:00.077052 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df858b4-efa3-433c-9dd0-00a502a3a3c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:00.077329 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df858b4-efa3-433c-9dd0-00a502a3a3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:00.077600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df858b4-efa3-433c-9dd0-00a502a3a3c5 None admin] PATH_INFO: `/v3/projects/9b2fa8031dc2446687372008e202c827/users/2d78b255174e4b84ab91333d261de646/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:00.079534 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df858b4-efa3-433c-9dd0-00a502a3a3c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=9b2fa8031dc2446687372008e202c827, user_id=2d78b255174e4b84ab91333d261de646, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:00.080945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df858b4-efa3-433c-9dd0-00a502a3a3c5 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:00.093221 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 597/1153] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:26:59 2022] PUT /identity/v3/projects/9b2fa8031dc2446687372008e202c827/users/2d78b255174e4b84ab91333d261de646/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:27:02.274597 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604376cd-85db-41ec-a496-280b6b25651d None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:02.275191 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604376cd-85db-41ec-a496-280b6b25651d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:02.275191 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604376cd-85db-41ec-a496-280b6b25651d None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:02.276548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 557/1154] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:27:02.281575 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea018aa2-639f-49c6-810c-c240802c5b15 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:02.281972 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea018aa2-639f-49c6-810c-c240802c5b15 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:02.282121 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea018aa2-639f-49c6-810c-c240802c5b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:02.298381 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-ea018aa2-639f-49c6-810c-c240802c5b15 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:27:02.299591 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ea018aa2-639f-49c6-810c-c240802c5b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:27:02.355540 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:02.364278 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 598/1155] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:27:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:27:02.388150 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604376cd-85db-41ec-a496-280b6b25651d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:02.486620 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604376cd-85db-41ec-a496-280b6b25651d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:02.518725 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20703d28-8411-409e-96b3-d1f9a240fed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:02.519505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20703d28-8411-409e-96b3-d1f9a240fed0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:02.519788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20703d28-8411-409e-96b3-d1f9a240fed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:02.520060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20703d28-8411-409e-96b3-d1f9a240fed0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:02.530910 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20703d28-8411-409e-96b3-d1f9a240fed0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:27:02.538354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-20703d28-8411-409e-96b3-d1f9a240fed0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:27:02.538354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:27:02.538354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:27:02.538354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:27:02.538354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:27:02.538354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:27:02.538354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:27:02.538354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:27:02.538354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:27:02.538354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:27:02.538354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:27:02.538354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:27:02.538354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:27:02.538354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:27:02.538354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:27:02.538354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:27:02.540218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:27:02.540218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:27:02.540218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:27:02.540218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:27:02.540218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:27:02.540218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:27:02.540218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:27:02.540218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:27:02.540218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:27:02.540218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:27:02.540218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:27:02.540218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:27:02.540218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:27:02.540218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:27:02.540218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:27:02.540218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:27:02.541416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:27:02.541416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:27:02.541416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:27:02.541416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:27:02.541416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:27:02.541416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:27:02.541416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:27:02.541416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:27:02.541416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:27:02.541416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:27:02.541416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:27:02.541416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:27:02.541416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 558/1156] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:27:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:27:02.545719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea018aa2-639f-49c6-810c-c240802c5b15 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:02.640538 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea018aa2-639f-49c6-810c-c240802c5b15 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:02.672715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abd7893-e65b-46f5-9d59-5260070ec271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:02.673370 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd7893-e65b-46f5-9d59-5260070ec271 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:02.673653 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd7893-e65b-46f5-9d59-5260070ec271 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:02.673929 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd7893-e65b-46f5-9d59-5260070ec271 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:02.674695 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abd7893-e65b-46f5-9d59-5260070ec271 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:27:02.675007 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abd7893-e65b-46f5-9d59-5260070ec271 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:02.676369 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abd7893-e65b-46f5-9d59-5260070ec271 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:02.684736 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 599/1157] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:27:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:27:02.690146 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20703d28-8411-409e-96b3-d1f9a240fed0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:02.722845 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:02.786466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20703d28-8411-409e-96b3-d1f9a240fed0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:02.819544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f9d94-7ccd-480c-932e-ee49d421ad48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:02.820431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f9d94-7ccd-480c-932e-ee49d421ad48 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:02.820715 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f9d94-7ccd-480c-932e-ee49d421ad48 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:02.820994 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f9d94-7ccd-480c-932e-ee49d421ad48 None admin] PATH_INFO: `/v3/users/2d78b255174e4b84ab91333d261de646` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:02.822163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f9d94-7ccd-480c-932e-ee49d421ad48 None admin] RBAC: Authorizing `identity:get_user(user_id=2d78b255174e4b84ab91333d261de646)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:02.823627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f9d94-7ccd-480c-932e-ee49d421ad48 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:02.825494 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 559/1158] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:27:02 2022] GET /identity/v3/users/2d78b255174e4b84ab91333d261de646 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:27:02.831335 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abd7893-e65b-46f5-9d59-5260070ec271 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:02.885750 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:02.931238 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abd7893-e65b-46f5-9d59-5260070ec271 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:02.964871 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d5b92-6a5f-4e08-b99f-a11f3252ce65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:02.965517 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d5b92-6a5f-4e08-b99f-a11f3252ce65 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:02.965805 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d5b92-6a5f-4e08-b99f-a11f3252ce65 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:02.966086 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d5b92-6a5f-4e08-b99f-a11f3252ce65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:03.030478 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6d5b92-6a5f-4e08-b99f-a11f3252ce65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:03.031960 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6d5b92-6a5f-4e08-b99f-a11f3252ce65 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:03.083551 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:03.127823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 600/1159] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:27:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 31 14:27:03.133517 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f9d94-7ccd-480c-932e-ee49d421ad48 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:03.230296 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f9d94-7ccd-480c-932e-ee49d421ad48 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:03.231732 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:03.264756 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e95cb2-b75f-4323-ac3d-154db6dc491d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:03.265616 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e95cb2-b75f-4323-ac3d-154db6dc491d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:03.265904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e95cb2-b75f-4323-ac3d-154db6dc491d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:03.266184 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e95cb2-b75f-4323-ac3d-154db6dc491d None admin] PATH_INFO: `/v3/projects/9b2fa8031dc2446687372008e202c827` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:03.267373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e95cb2-b75f-4323-ac3d-154db6dc491d None admin] RBAC: Authorizing `identity:get_project(project_id=9b2fa8031dc2446687372008e202c827)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:03.268789 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e95cb2-b75f-4323-ac3d-154db6dc491d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:03.270480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 560/1160] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:27:03 2022] GET /identity/v3/projects/9b2fa8031dc2446687372008e202c827 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:27:03.277076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d5b92-6a5f-4e08-b99f-a11f3252ce65 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:03.371739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d5b92-6a5f-4e08-b99f-a11f3252ce65 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:03.404739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1dada1-e1f0-47c6-905c-4c0d4d12e51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:03.405381 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dada1-e1f0-47c6-905c-4c0d4d12e51a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:03.405692 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dada1-e1f0-47c6-905c-4c0d4d12e51a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:03.405970 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dada1-e1f0-47c6-905c-4c0d4d12e51a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:03.406827 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1dada1-e1f0-47c6-905c-4c0d4d12e51a None admin] RBAC: Adding query filter params (role.id=e8d679596ffb45939229d1e2a13ce9b0, scope.project.id=9b2fa8031dc2446687372008e202c827, user.id=2d78b255174e4b84ab91333d261de646) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:27:03.407143 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1dada1-e1f0-47c6-905c-4c0d4d12e51a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:03.408558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1dada1-e1f0-47c6-905c-4c0d4d12e51a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:03.414239 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 601/1161] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:27:03 2022] GET /identity/v3/role_assignments?user.id=2d78b255174e4b84ab91333d261de646&scope.project.id=9b2fa8031dc2446687372008e202c827&role.id=e8d679596ffb45939229d1e2a13ce9b0 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:27:05.635035 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114ea7d-9e27-43d1-8e48-9a4cf4a8aa99 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:05.635567 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114ea7d-9e27-43d1-8e48-9a4cf4a8aa99 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:05.635668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114ea7d-9e27-43d1-8e48-9a4cf4a8aa99 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:05.636980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 561/1162] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:27:05.642100 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07924c-118a-4d92-bb39-d83bf2224d07 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:05.642531 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07924c-118a-4d92-bb39-d83bf2224d07 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:05.642674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07924c-118a-4d92-bb39-d83bf2224d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:05.659628 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-7e07924c-118a-4d92-bb39-d83bf2224d07 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:27:05.660815 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7e07924c-118a-4d92-bb39-d83bf2224d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:27:05.674227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:05.725387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 602/1163] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:27:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:27:05.747035 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114ea7d-9e27-43d1-8e48-9a4cf4a8aa99 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:05.844470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114ea7d-9e27-43d1-8e48-9a4cf4a8aa99 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:05.876983 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab19330-0e10-4e00-b340-42e27f80d620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:05.877617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab19330-0e10-4e00-b340-42e27f80d620 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:05.877905 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab19330-0e10-4e00-b340-42e27f80d620 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:05.878184 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab19330-0e10-4e00-b340-42e27f80d620 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:05.915577 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:05.943985 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab19330-0e10-4e00-b340-42e27f80d620 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:05.946040 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab19330-0e10-4e00-b340-42e27f80d620 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:06.040933 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 562/1164] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:27:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 31 14:27:06.047051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07924c-118a-4d92-bb39-d83bf2224d07 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:06.141632 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07924c-118a-4d92-bb39-d83bf2224d07 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:06.173870 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a84c5-e951-4faf-9f4c-a78bce28a8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:06.174598 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a84c5-e951-4faf-9f4c-a78bce28a8cb None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:06.174914 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a84c5-e951-4faf-9f4c-a78bce28a8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:06.175188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a84c5-e951-4faf-9f4c-a78bce28a8cb None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:06.182107 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a84c5-e951-4faf-9f4c-a78bce28a8cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:06.183572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a84c5-e951-4faf-9f4c-a78bce28a8cb None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:06.197923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-895a84c5-e951-4faf-9f4c-a78bce28a8cb None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 31 14:27:06.197923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:27:06.197923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 31 14:27:06.197923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 31 14:27:06.197923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:27:06.197923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:27:06.197923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:27:06.197923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:27:06.197923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 31 14:27:06.197923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:27:06.197923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:27:06.197923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:27:06.197923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:27:06.197923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:27:06.197923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:27:06.197923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:27:06.197923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:27:06.199190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:27:06.199190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:27:06.199190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:27:06.199190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:27:06.199190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:27:06.199190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:27:06.199190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:27:06.199190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:27:06.199190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:27:06.199190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:27:06.199190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:27:06.199190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:27:06.199190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:27:06.199190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:27:06.199190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:27:06.199190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:27:06.200318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:27:06.200318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:27:06.200318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:27:06.200318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:27:06.200318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:27:06.200318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:27:06.200318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:27:06.200318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:27:06.200318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:27:06.200318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:27:06.200318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 31 14:27:06.200318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:27:06.200318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 31 14:27:06.200318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:27:06.200318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 603/1165] 10.222.0.67 () {56 vars in 1117 bytes} [Tue May 31 14:27:06 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:27:06.205363 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab19330-0e10-4e00-b340-42e27f80d620 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:06.271898 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:06.302570 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab19330-0e10-4e00-b340-42e27f80d620 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:06.335955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337745bc-6a63-4a4a-9912-132c6e1d5729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:06.336609 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337745bc-6a63-4a4a-9912-132c6e1d5729 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:06.336897 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337745bc-6a63-4a4a-9912-132c6e1d5729 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:06.337179 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337745bc-6a63-4a4a-9912-132c6e1d5729 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:06.337897 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337745bc-6a63-4a4a-9912-132c6e1d5729 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:27:06.338209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337745bc-6a63-4a4a-9912-132c6e1d5729 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:06.339605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337745bc-6a63-4a4a-9912-132c6e1d5729 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:06.346605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 563/1166] 10.222.0.67 () {56 vars in 1115 bytes} [Tue May 31 14:27:06 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:27:08.537715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c19845-dbc4-4b46-8397-7f4a56ba71cd None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:08.538226 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c19845-dbc4-4b46-8397-7f4a56ba71cd None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:08.538226 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c19845-dbc4-4b46-8397-7f4a56ba71cd None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:08.539578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 604/1167] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:27:08.544819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36608e29-07c7-46a1-b4be-4ba85c94d614 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:08.545193 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36608e29-07c7-46a1-b4be-4ba85c94d614 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:08.545388 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36608e29-07c7-46a1-b4be-4ba85c94d614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:08.564236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-36608e29-07c7-46a1-b4be-4ba85c94d614 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:27:08.565441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-36608e29-07c7-46a1-b4be-4ba85c94d614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:27:08.629540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 564/1168] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:27:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:27:08.652780 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c19845-dbc4-4b46-8397-7f4a56ba71cd None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:08.665244 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:08.750469 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c19845-dbc4-4b46-8397-7f4a56ba71cd None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:08.782977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55122dd4-fdbc-4a7a-9f6b-667baeb2cc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:08.783610 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55122dd4-fdbc-4a7a-9f6b-667baeb2cc3c None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:08.783889 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55122dd4-fdbc-4a7a-9f6b-667baeb2cc3c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:08.784161 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55122dd4-fdbc-4a7a-9f6b-667baeb2cc3c None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:08.784888 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55122dd4-fdbc-4a7a-9f6b-667baeb2cc3c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:08.786236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55122dd4-fdbc-4a7a-9f6b-667baeb2cc3c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:08.799918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 605/1169] 10.222.0.67 () {60 vars in 1138 bytes} [Tue May 31 14:27:08 2022] POST /identity/v3/domains => generated 267 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:27:11.039382 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785d507-96be-4b06-bc76-d7e33acf0fde None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:11.039948 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785d507-96be-4b06-bc76-d7e33acf0fde None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:11.039948 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785d507-96be-4b06-bc76-d7e33acf0fde None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:11.041299 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 565/1170] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:27:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:27:11.046629 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f0061-e574-435e-a1c6-de5fa9da0d75 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:11.047017 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f0061-e574-435e-a1c6-de5fa9da0d75 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:11.047221 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f0061-e574-435e-a1c6-de5fa9da0d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:11.064926 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-7b6f0061-e574-435e-a1c6-de5fa9da0d75 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:27:11.066121 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7b6f0061-e574-435e-a1c6-de5fa9da0d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:27:11.137783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:11.143632 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 606/1171] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:27:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:27:11.167973 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785d507-96be-4b06-bc76-d7e33acf0fde None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:11.228304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:11.264471 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785d507-96be-4b06-bc76-d7e33acf0fde None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:11.297071 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a963516-1e87-4b91-968c-34656f1ee9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:11.297863 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a963516-1e87-4b91-968c-34656f1ee9bc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:11.298146 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a963516-1e87-4b91-968c-34656f1ee9bc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:11.298422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a963516-1e87-4b91-968c-34656f1ee9bc None admin] PATH_INFO: `/v3/domains/bf3ef8a1edad460197899dc5987a0d66` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:11.308547 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a963516-1e87-4b91-968c-34656f1ee9bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=bf3ef8a1edad460197899dc5987a0d66)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:11.309987 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a963516-1e87-4b91-968c-34656f1ee9bc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:11.311606 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 566/1172] 10.222.0.67 () {56 vars in 1183 bytes} [Tue May 31 14:27:11 2022] GET /identity/v3/domains/bf3ef8a1edad460197899dc5987a0d66 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:27:11.317362 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f0061-e574-435e-a1c6-de5fa9da0d75 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:11.412586 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f0061-e574-435e-a1c6-de5fa9da0d75 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:11.445419 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6dbc7-dd7d-47f1-8ca8-a91c56616676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:11.446068 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6dbc7-dd7d-47f1-8ca8-a91c56616676 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:11.446358 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6dbc7-dd7d-47f1-8ca8-a91c56616676 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:11.446635 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6dbc7-dd7d-47f1-8ca8-a91c56616676 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:11.447367 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6dbc7-dd7d-47f1-8ca8-a91c56616676 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:11.448814 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6dbc7-dd7d-47f1-8ca8-a91c56616676 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:11.475682 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 607/1173] 10.222.0.67 () {60 vars in 1141 bytes} [Tue May 31 14:27:11 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:27:13.727562 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f767ac-9ebe-4c88-9a4f-4d2cd8e59a98 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:13.728105 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f767ac-9ebe-4c88-9a4f-4d2cd8e59a98 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:13.728105 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f767ac-9ebe-4c88-9a4f-4d2cd8e59a98 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:13.729480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 567/1174] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:27:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:27:13.734790 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b7430-f769-4b69-ac45-77d684ad1110 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:13.735235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b7430-f769-4b69-ac45-77d684ad1110 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:13.735340 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b7430-f769-4b69-ac45-77d684ad1110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:13.752281 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-fd9b7430-f769-4b69-ac45-77d684ad1110 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:27:13.753463 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-fd9b7430-f769-4b69-ac45-77d684ad1110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:27:13.801738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:13.823032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 608/1175] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:27:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:27:13.847632 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f767ac-9ebe-4c88-9a4f-4d2cd8e59a98 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:13.946913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f767ac-9ebe-4c88-9a4f-4d2cd8e59a98 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:13.968966 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:13.980243 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c2727-ee3a-4cf6-92f9-7945ef51c3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:13.981039 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c2727-ee3a-4cf6-92f9-7945ef51c3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:13.981320 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c2727-ee3a-4cf6-92f9-7945ef51c3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:13.981592 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c2727-ee3a-4cf6-92f9-7945ef51c3d9 None admin] PATH_INFO: `/v3/domains/bf3ef8a1edad460197899dc5987a0d66` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:13.982787 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c2727-ee3a-4cf6-92f9-7945ef51c3d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=bf3ef8a1edad460197899dc5987a0d66)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:13.984208 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c2727-ee3a-4cf6-92f9-7945ef51c3d9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:13.985800 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 568/1176] 10.222.0.67 () {56 vars in 1183 bytes} [Tue May 31 14:27:13 2022] GET /identity/v3/domains/bf3ef8a1edad460197899dc5987a0d66 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:27:13.991407 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b7430-f769-4b69-ac45-77d684ad1110 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:14.085157 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b7430-f769-4b69-ac45-77d684ad1110 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:14.119559 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e9fed-7b40-49d7-8e18-047552a75694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:14.120239 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e9fed-7b40-49d7-8e18-047552a75694 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:14.120525 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e9fed-7b40-49d7-8e18-047552a75694 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:14.120805 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e9fed-7b40-49d7-8e18-047552a75694 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:14.121515 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e9fed-7b40-49d7-8e18-047552a75694 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:14.122975 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e9fed-7b40-49d7-8e18-047552a75694 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:14.142815 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 609/1177] 10.222.0.67 () {60 vars in 1132 bytes} [Tue May 31 14:27:13 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:27:16.405418 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749105c3-8d8a-4867-9631-4e0ad25f2ad8 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:16.405918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749105c3-8d8a-4867-9631-4e0ad25f2ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:16.405918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749105c3-8d8a-4867-9631-4e0ad25f2ad8 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:16.407349 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 569/1178] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:27:16.412591 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d4440-5772-4768-94c1-8d85bd8eb9ad None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:16.413022 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d4440-5772-4768-94c1-8d85bd8eb9ad None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:16.413177 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d4440-5772-4768-94c1-8d85bd8eb9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:16.430007 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-337d4440-5772-4768-94c1-8d85bd8eb9ad None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:27:16.431277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-337d4440-5772-4768-94c1-8d85bd8eb9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:27:16.460222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:16.497942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 610/1179] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:27:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:27:16.522070 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749105c3-8d8a-4867-9631-4e0ad25f2ad8 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:16.619269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749105c3-8d8a-4867-9631-4e0ad25f2ad8 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:16.650880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff2324-b348-4c8a-b741-046b9d39c69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:16.651651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff2324-b348-4c8a-b741-046b9d39c69d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:16.651934 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff2324-b348-4c8a-b741-046b9d39c69d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:16.652209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff2324-b348-4c8a-b741-046b9d39c69d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:16.662375 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff2324-b348-4c8a-b741-046b9d39c69d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:27:16.669802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-30ff2324-b348-4c8a-b741-046b9d39c69d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:27:16.669802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:27:16.669802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:27:16.669802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:27:16.669802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:27:16.669802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:27:16.669802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:27:16.669802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:27:16.669802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:27:16.669802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:27:16.669802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:27:16.669802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:27:16.669802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:27:16.669802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:27:16.669802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:27:16.669802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:27:16.670947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:27:16.670947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:27:16.670947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:27:16.670947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:27:16.670947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:27:16.670947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:27:16.670947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:27:16.670947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:27:16.670947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:27:16.670947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:27:16.670947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:27:16.670947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:27:16.670947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:27:16.670947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:27:16.670947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:27:16.670947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:27:16.672020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:27:16.672020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:27:16.672020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:27:16.672020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:27:16.672020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:27:16.672020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:27:16.672020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:27:16.672020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:27:16.672020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:27:16.672020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:27:16.672020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:27:16.672020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:27:16.672020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 570/1180] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:27:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:27:16.677888 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d4440-5772-4768-94c1-8d85bd8eb9ad None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:16.772593 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d4440-5772-4768-94c1-8d85bd8eb9ad None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:16.805214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afaa01e-f070-4186-9929-25cff0eb8e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:16.805856 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afaa01e-f070-4186-9929-25cff0eb8e19 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:16.806134 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afaa01e-f070-4186-9929-25cff0eb8e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:16.806405 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afaa01e-f070-4186-9929-25cff0eb8e19 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:16.807168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afaa01e-f070-4186-9929-25cff0eb8e19 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:27:16.807473 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afaa01e-f070-4186-9929-25cff0eb8e19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:16.808840 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afaa01e-f070-4186-9929-25cff0eb8e19 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:16.816597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 611/1181] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:27:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:27:16.822578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff2324-b348-4c8a-b741-046b9d39c69d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:16.872899 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:16.919873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff2324-b348-4c8a-b741-046b9d39c69d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:16.952900 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11abb8-52c5-41f9-8f56-5559fcf9e387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:16.953752 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11abb8-52c5-41f9-8f56-5559fcf9e387 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:16.954039 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11abb8-52c5-41f9-8f56-5559fcf9e387 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:16.954312 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11abb8-52c5-41f9-8f56-5559fcf9e387 None admin] PATH_INFO: `/v3/users/f6a9b7b5a6ca415493f510255bdd5338` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:16.966817 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d11abb8-52c5-41f9-8f56-5559fcf9e387 None admin] RBAC: Authorizing `identity:get_user(user_id=f6a9b7b5a6ca415493f510255bdd5338)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:16.968236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d11abb8-52c5-41f9-8f56-5559fcf9e387 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:16.969846 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 571/1182] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:27:16 2022] GET /identity/v3/users/f6a9b7b5a6ca415493f510255bdd5338 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:27:16.975483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afaa01e-f070-4186-9929-25cff0eb8e19 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:17.006318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:17.072820 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afaa01e-f070-4186-9929-25cff0eb8e19 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:17.105375 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc33ce-b2df-440a-8efa-4ba4d82968cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:17.106007 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc33ce-b2df-440a-8efa-4ba4d82968cd None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:17.106289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc33ce-b2df-440a-8efa-4ba4d82968cd None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:17.106562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc33ce-b2df-440a-8efa-4ba4d82968cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:17.170323 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fc33ce-b2df-440a-8efa-4ba4d82968cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:17.171738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fc33ce-b2df-440a-8efa-4ba4d82968cd None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:17.198467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:17.264734 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 612/1183] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:27:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 31 14:27:17.270584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11abb8-52c5-41f9-8f56-5559fcf9e387 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:17.366699 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11abb8-52c5-41f9-8f56-5559fcf9e387 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:17.376786 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:17.399708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e74300-55dd-4e09-be02-bee892aafb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:17.400546 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e74300-55dd-4e09-be02-bee892aafb8f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:17.400830 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e74300-55dd-4e09-be02-bee892aafb8f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:17.401105 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e74300-55dd-4e09-be02-bee892aafb8f None admin] PATH_INFO: `/v3/projects/f05f064fe74548bda2b44934493525cf` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:17.402262 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e74300-55dd-4e09-be02-bee892aafb8f None admin] RBAC: Authorizing `identity:get_project(project_id=f05f064fe74548bda2b44934493525cf)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:17.403711 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e74300-55dd-4e09-be02-bee892aafb8f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:17.405371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 572/1184] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:27:17 2022] GET /identity/v3/projects/f05f064fe74548bda2b44934493525cf => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:27:17.411525 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc33ce-b2df-440a-8efa-4ba4d82968cd None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:17.505525 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc33ce-b2df-440a-8efa-4ba4d82968cd None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:17.536010 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:17.542115 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa83c19-f366-4563-b4f8-83ae66b24d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:17.542780 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa83c19-f366-4563-b4f8-83ae66b24d11 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:17.543061 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa83c19-f366-4563-b4f8-83ae66b24d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:17.543334 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa83c19-f366-4563-b4f8-83ae66b24d11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:17.544159 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa83c19-f366-4563-b4f8-83ae66b24d11 None admin] RBAC: Adding query filter params (role.id=e8d679596ffb45939229d1e2a13ce9b0, scope.project.id=f05f064fe74548bda2b44934493525cf, user.id=f6a9b7b5a6ca415493f510255bdd5338) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:27:17.544468 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa83c19-f366-4563-b4f8-83ae66b24d11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:17.545896 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa83c19-f366-4563-b4f8-83ae66b24d11 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:17.551210 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 613/1185] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:27:17 2022] GET /identity/v3/role_assignments?user.id=f6a9b7b5a6ca415493f510255bdd5338&scope.project.id=f05f064fe74548bda2b44934493525cf&role.id=e8d679596ffb45939229d1e2a13ce9b0 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:27:19.737109 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3afc57-849f-451a-b7ff-3d25058ac464 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:19.737656 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3afc57-849f-451a-b7ff-3d25058ac464 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:19.737656 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3afc57-849f-451a-b7ff-3d25058ac464 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:19.739066 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 573/1186] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:27:19.744781 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd27f6-3ac4-43c8-a147-56974ccf5b89 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:19.745155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd27f6-3ac4-43c8-a147-56974ccf5b89 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:19.745331 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd27f6-3ac4-43c8-a147-56974ccf5b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:19.761505 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-41cd27f6-3ac4-43c8-a147-56974ccf5b89 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:27:19.762708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-41cd27f6-3ac4-43c8-a147-56974ccf5b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:27:19.826295 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 614/1187] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:27:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:27:19.848305 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3afc57-849f-451a-b7ff-3d25058ac464 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:19.946882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3afc57-849f-451a-b7ff-3d25058ac464 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:19.980078 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6051a-ebdc-4d2f-8f15-f99a44ba6829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:19.980859 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6051a-ebdc-4d2f-8f15-f99a44ba6829 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:19.981146 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6051a-ebdc-4d2f-8f15-f99a44ba6829 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:19.981428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6051a-ebdc-4d2f-8f15-f99a44ba6829 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:19.991382 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b6051a-ebdc-4d2f-8f15-f99a44ba6829 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:27:19.999510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-29b6051a-ebdc-4d2f-8f15-f99a44ba6829 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:27:19.999510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:27:19.999510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:27:19.999510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:27:19.999510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:27:19.999510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:27:19.999510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:27:19.999510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:27:19.999510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:27:19.999510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:27:19.999510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:27:19.999510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:27:19.999510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:27:19.999510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:27:19.999510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:27:19.999510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:27:20.000607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:27:20.000607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:27:20.000607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:27:20.000607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:27:20.000607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:27:20.000607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:27:20.000607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:27:20.000607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:27:20.000607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:27:20.000607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:27:20.000607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:27:20.000607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:27:20.000607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:27:20.000607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:27:20.000607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:27:20.000607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:27:20.001644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:27:20.001644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:27:20.001644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:27:20.001644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:27:20.001644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:27:20.001644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:27:20.001644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:27:20.001644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:27:20.001644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:27:20.001644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:27:20.001644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:27:20.001644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:27:20.001644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 574/1188] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:27:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:27:20.006534 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd27f6-3ac4-43c8-a147-56974ccf5b89 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:20.100671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd27f6-3ac4-43c8-a147-56974ccf5b89 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:20.124946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:20.136423 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851037c-3c48-460b-9142-8573a91ec72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:20.137090 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851037c-3c48-460b-9142-8573a91ec72e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:20.137377 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851037c-3c48-460b-9142-8573a91ec72e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:20.137656 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851037c-3c48-460b-9142-8573a91ec72e None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:20.138405 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c851037c-3c48-460b-9142-8573a91ec72e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:27:20.138753 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c851037c-3c48-460b-9142-8573a91ec72e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:20.140118 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c851037c-3c48-460b-9142-8573a91ec72e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:20.148566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 615/1189] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:27:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:27:20.154323 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6051a-ebdc-4d2f-8f15-f99a44ba6829 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:20.193322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:20.252648 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6051a-ebdc-4d2f-8f15-f99a44ba6829 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:20.284595 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdfe9f3-c7a1-48c0-a488-ded5a42c3a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:20.285449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfe9f3-c7a1-48c0-a488-ded5a42c3a37 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:20.285732 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfe9f3-c7a1-48c0-a488-ded5a42c3a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:20.286004 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfe9f3-c7a1-48c0-a488-ded5a42c3a37 None admin] PATH_INFO: `/v3/users/f6a9b7b5a6ca415493f510255bdd5338` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:20.287193 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdfe9f3-c7a1-48c0-a488-ded5a42c3a37 None admin] RBAC: Authorizing `identity:get_user(user_id=f6a9b7b5a6ca415493f510255bdd5338)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:20.288602 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdfe9f3-c7a1-48c0-a488-ded5a42c3a37 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:20.290199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 575/1190] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:27:20 2022] GET /identity/v3/users/f6a9b7b5a6ca415493f510255bdd5338 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:27:20.295722 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851037c-3c48-460b-9142-8573a91ec72e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:20.393856 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851037c-3c48-460b-9142-8573a91ec72e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:20.425854 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d695b-8e34-453e-9849-2fc57a2d9d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:20.426734 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d695b-8e34-453e-9849-2fc57a2d9d10 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:20.427021 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d695b-8e34-453e-9849-2fc57a2d9d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:20.427300 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d695b-8e34-453e-9849-2fc57a2d9d10 None admin] PATH_INFO: `/v3/projects/f05f064fe74548bda2b44934493525cf` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:20.428464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d695b-8e34-453e-9849-2fc57a2d9d10 None admin] RBAC: Authorizing `identity:get_project(project_id=f05f064fe74548bda2b44934493525cf)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:20.430402 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d695b-8e34-453e-9849-2fc57a2d9d10 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:20.432100 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 616/1191] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:27:20 2022] GET /identity/v3/projects/f05f064fe74548bda2b44934493525cf => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:27:20.437678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdfe9f3-c7a1-48c0-a488-ded5a42c3a37 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:20.533387 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdfe9f3-c7a1-48c0-a488-ded5a42c3a37 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:20.540324 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:20.566300 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724acf10-39e5-4b1e-8cdc-fabf3e06dff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:20.567201 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724acf10-39e5-4b1e-8cdc-fabf3e06dff7 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:20.567486 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724acf10-39e5-4b1e-8cdc-fabf3e06dff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:20.567761 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724acf10-39e5-4b1e-8cdc-fabf3e06dff7 None admin] PATH_INFO: `/v3/projects/f05f064fe74548bda2b44934493525cf/users/f6a9b7b5a6ca415493f510255bdd5338/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:20.570168 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724acf10-39e5-4b1e-8cdc-fabf3e06dff7 None admin] RBAC: Authorizing `identity:create_grant(project_id=f05f064fe74548bda2b44934493525cf, user_id=f6a9b7b5a6ca415493f510255bdd5338, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:20.571577 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724acf10-39e5-4b1e-8cdc-fabf3e06dff7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:20.583592 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 576/1192] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:27:20 2022] PUT /identity/v3/projects/f05f064fe74548bda2b44934493525cf/users/f6a9b7b5a6ca415493f510255bdd5338/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:27:22.787626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9c4fb-fae2-4c4a-a0d7-329bc455aa2c None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:22.788640 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9c4fb-fae2-4c4a-a0d7-329bc455aa2c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:22.788735 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9c4fb-fae2-4c4a-a0d7-329bc455aa2c None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:22.790039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 617/1193] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:27:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:27:22.795206 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88a41a-cb7b-4868-93ad-a37e2fc466e3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:22.795573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88a41a-cb7b-4868-93ad-a37e2fc466e3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:22.795775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88a41a-cb7b-4868-93ad-a37e2fc466e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:22.812792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ef88a41a-cb7b-4868-93ad-a37e2fc466e3 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:27:22.813980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ef88a41a-cb7b-4868-93ad-a37e2fc466e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:27:22.878362 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 577/1194] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:27:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:27:22.901366 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9c4fb-fae2-4c4a-a0d7-329bc455aa2c None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:22.957934 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:23.002835 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9c4fb-fae2-4c4a-a0d7-329bc455aa2c None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:23.034564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239e648-805e-4b5f-9e7c-84d0eeb4b67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:23.035367 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239e648-805e-4b5f-9e7c-84d0eeb4b67e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:23.035651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239e648-805e-4b5f-9e7c-84d0eeb4b67e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:23.035928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239e648-805e-4b5f-9e7c-84d0eeb4b67e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:23.046771 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239e648-805e-4b5f-9e7c-84d0eeb4b67e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:27:23.054866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-c239e648-805e-4b5f-9e7c-84d0eeb4b67e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:27:23.054866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:27:23.054866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:27:23.054866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:27:23.054866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:27:23.054866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:27:23.054866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:27:23.054866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:27:23.054866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:27:23.054866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:27:23.054866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:27:23.054866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:27:23.054866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:27:23.054866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:27:23.054866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:27:23.054866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:27:23.055801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:27:23.055801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:27:23.055801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:27:23.055801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:27:23.055801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:27:23.055801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:27:23.055801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:27:23.055801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:27:23.055801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:27:23.055801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:27:23.055801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:27:23.055801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:27:23.055801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:27:23.055801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:27:23.055801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:27:23.055801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:27:23.056658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:27:23.056658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:27:23.056658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:27:23.056658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:27:23.056658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:27:23.056658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:27:23.056658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:27:23.056658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:27:23.056658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:27:23.056658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:27:23.056658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:27:23.056658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:27:23.056658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 618/1195] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:27:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:27:23.062099 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88a41a-cb7b-4868-93ad-a37e2fc466e3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:23.152745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:23.159219 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88a41a-cb7b-4868-93ad-a37e2fc466e3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:23.191887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b144879-2cde-4f2c-a254-3868584d33c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:23.192543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b144879-2cde-4f2c-a254-3868584d33c2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:23.192827 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b144879-2cde-4f2c-a254-3868584d33c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:23.193103 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b144879-2cde-4f2c-a254-3868584d33c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:23.193839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b144879-2cde-4f2c-a254-3868584d33c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:27:23.194146 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b144879-2cde-4f2c-a254-3868584d33c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:23.195564 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b144879-2cde-4f2c-a254-3868584d33c2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:23.203656 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 578/1196] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:27:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:27:23.209367 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239e648-805e-4b5f-9e7c-84d0eeb4b67e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:23.304291 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239e648-805e-4b5f-9e7c-84d0eeb4b67e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:23.317327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:23.337873 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d169c3-1074-47c5-9ee4-c92a101682a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:23.338759 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d169c3-1074-47c5-9ee4-c92a101682a8 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:23.339045 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d169c3-1074-47c5-9ee4-c92a101682a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:23.339319 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d169c3-1074-47c5-9ee4-c92a101682a8 None admin] PATH_INFO: `/v3/users/f6a9b7b5a6ca415493f510255bdd5338` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:23.340498 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d169c3-1074-47c5-9ee4-c92a101682a8 None admin] RBAC: Authorizing `identity:get_user(user_id=f6a9b7b5a6ca415493f510255bdd5338)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:23.341928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d169c3-1074-47c5-9ee4-c92a101682a8 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:23.343569 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 619/1197] 10.222.0.67 () {56 vars in 1177 bytes} [Tue May 31 14:27:23 2022] GET /identity/v3/users/f6a9b7b5a6ca415493f510255bdd5338 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:27:23.349044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b144879-2cde-4f2c-a254-3868584d33c2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:23.445210 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b144879-2cde-4f2c-a254-3868584d33c2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:23.478234 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d3148-4409-49f2-bcf2-4f0e8c2df56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:23.478893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d3148-4409-49f2-bcf2-4f0e8c2df56a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:23.479180 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d3148-4409-49f2-bcf2-4f0e8c2df56a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:23.479459 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d3148-4409-49f2-bcf2-4f0e8c2df56a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:23.504519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:23.545846 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d3148-4409-49f2-bcf2-4f0e8c2df56a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:23.547319 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d3148-4409-49f2-bcf2-4f0e8c2df56a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:23.641917 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 579/1198] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:27:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 31 14:27:23.648407 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d169c3-1074-47c5-9ee4-c92a101682a8 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:23.742521 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d169c3-1074-47c5-9ee4-c92a101682a8 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:23.774941 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6693c32-a33b-4c19-8b95-e8e98970ba51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:23.775778 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6693c32-a33b-4c19-8b95-e8e98970ba51 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:23.776056 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6693c32-a33b-4c19-8b95-e8e98970ba51 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:23.776324 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6693c32-a33b-4c19-8b95-e8e98970ba51 None admin] PATH_INFO: `/v3/projects/f05f064fe74548bda2b44934493525cf` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:23.777471 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6693c32-a33b-4c19-8b95-e8e98970ba51 None admin] RBAC: Authorizing `identity:get_project(project_id=f05f064fe74548bda2b44934493525cf)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:23.778914 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6693c32-a33b-4c19-8b95-e8e98970ba51 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:23.780572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 620/1199] 10.222.0.67 () {56 vars in 1186 bytes} [Tue May 31 14:27:23 2022] GET /identity/v3/projects/f05f064fe74548bda2b44934493525cf => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:27:23.786517 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d3148-4409-49f2-bcf2-4f0e8c2df56a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:27:23.842318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:23.882679 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d3148-4409-49f2-bcf2-4f0e8c2df56a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:27:23.915174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3fdbfa-a120-40ab-a470-cc611176efee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:27:23.915758 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fdbfa-a120-40ab-a470-cc611176efee None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:27:23.916043 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fdbfa-a120-40ab-a470-cc611176efee None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:27:23.916320 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fdbfa-a120-40ab-a470-cc611176efee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:27:23.917136 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3fdbfa-a120-40ab-a470-cc611176efee None admin] RBAC: Adding query filter params (role.id=e8d679596ffb45939229d1e2a13ce9b0, scope.project.id=f05f064fe74548bda2b44934493525cf, user.id=f6a9b7b5a6ca415493f510255bdd5338) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:27:23.917450 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3fdbfa-a120-40ab-a470-cc611176efee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:27:23.918881 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3fdbfa-a120-40ab-a470-cc611176efee None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:27:23.924580 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 580/1200] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:27:23 2022] GET /identity/v3/role_assignments?user.id=f6a9b7b5a6ca415493f510255bdd5338&scope.project.id=f05f064fe74548bda2b44934493525cf&role.id=e8d679596ffb45939229d1e2a13ce9b0 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:27:33.770393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=75 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:27:33.921927 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=39 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:25.255179 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be18100b-e2bf-4503-8537-e4e719b5f1ae None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:25.255794 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be18100b-e2bf-4503-8537-e4e719b5f1ae None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:25.255794 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be18100b-e2bf-4503-8537-e4e719b5f1ae None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:25.257074 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 621/1201] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:28:25.262697 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e123b-ae63-484f-9b09-2c99acfbd186 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:25.263588 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e123b-ae63-484f-9b09-2c99acfbd186 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:25.264083 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e123b-ae63-484f-9b09-2c99acfbd186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:25.287200 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-df5e123b-ae63-484f-9b09-2c99acfbd186 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:28:25.289147 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-df5e123b-ae63-484f-9b09-2c99acfbd186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:28:25.379745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 581/1202] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:28:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:28:25.405233 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be18100b-e2bf-4503-8537-e4e719b5f1ae None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:25.531820 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be18100b-e2bf-4503-8537-e4e719b5f1ae None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:25.579130 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907facbd-0b63-480d-9b54-707767acce18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:25.579416 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907facbd-0b63-480d-9b54-707767acce18 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:25.579658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907facbd-0b63-480d-9b54-707767acce18 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:25.579906 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907facbd-0b63-480d-9b54-707767acce18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:25.592337 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907facbd-0b63-480d-9b54-707767acce18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:25.593866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907facbd-0b63-480d-9b54-707767acce18 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:25.616627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-907facbd-0b63-480d-9b54-707767acce18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:25.616627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:25.616627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:25.616627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:25.616627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:25.616627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:25.616627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:25.616627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:25.616627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:25.616627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:25.616627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:25.616627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:25.616627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:28:25.616627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:28:25.616627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:28:25.618136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:28:25.618136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:25.618136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:25.618136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:25.618136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:25.618136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:25.618136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:25.618136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:25.618136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:28:25.618136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:25.618136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:25.618136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:25.618136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:25.618136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:25.618136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:25.618136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:25.618136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:25.619182 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:28:25.619182 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:28:25.619182 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:25.619182 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:28:25.619182 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 622/1203] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:28:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:25.624896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e123b-ae63-484f-9b09-2c99acfbd186 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:25.762923 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e123b-ae63-484f-9b09-2c99acfbd186 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:25.800092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:25.809821 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede03d7-4ea0-467d-ab71-1e623f9a0860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:25.810631 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede03d7-4ea0-467d-ab71-1e623f9a0860 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:25.811033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede03d7-4ea0-467d-ab71-1e623f9a0860 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:25.811433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede03d7-4ea0-467d-ab71-1e623f9a0860 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:25.812281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede03d7-4ea0-467d-ab71-1e623f9a0860 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:25.812600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede03d7-4ea0-467d-ab71-1e623f9a0860 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:25.814290 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede03d7-4ea0-467d-ab71-1e623f9a0860 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:25.832072 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 582/1204] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:28:25.837951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907facbd-0b63-480d-9b54-707767acce18 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:25.911075 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:25.975422 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907facbd-0b63-480d-9b54-707767acce18 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:26.019055 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24879e-789c-40c9-9530-eee4e2eef477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:26.019706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24879e-789c-40c9-9530-eee4e2eef477 None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:26.020648 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24879e-789c-40c9-9530-eee4e2eef477 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:26.020988 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24879e-789c-40c9-9530-eee4e2eef477 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:26.023248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a24879e-789c-40c9-9530-eee4e2eef477 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:26.024707 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a24879e-789c-40c9-9530-eee4e2eef477 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:26.048366 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 623/1205] 10.222.0.67 () {60 vars in 1132 bytes} [Tue May 31 14:28:25 2022] POST /identity/v3/users => generated 251 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:28:28.528149 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989067d-1cd1-44f8-a626-2a5ff6b36918 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:28.528698 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989067d-1cd1-44f8-a626-2a5ff6b36918 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:28.528698 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989067d-1cd1-44f8-a626-2a5ff6b36918 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:28.530004 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 583/1206] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:28:28.536770 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a9158-dfad-458a-8e56-1a5d2a4f2722 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:28.537279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a9158-dfad-458a-8e56-1a5d2a4f2722 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:28.537360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a9158-dfad-458a-8e56-1a5d2a4f2722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:28.558704 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-616a9158-dfad-458a-8e56-1a5d2a4f2722 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:28:28.560049 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-616a9158-dfad-458a-8e56-1a5d2a4f2722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:28:28.648555 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 624/1207] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:28:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:28:28.675603 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989067d-1cd1-44f8-a626-2a5ff6b36918 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:28.813029 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989067d-1cd1-44f8-a626-2a5ff6b36918 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:28.864943 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301340dc-a9e5-4d22-862c-885e464ff9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:28.865800 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301340dc-a9e5-4d22-862c-885e464ff9cb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:28.866085 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301340dc-a9e5-4d22-862c-885e464ff9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:28.866367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301340dc-a9e5-4d22-862c-885e464ff9cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:28.890880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301340dc-a9e5-4d22-862c-885e464ff9cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:28:28.899519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-301340dc-a9e5-4d22-862c-885e464ff9cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 31 14:28:28.899519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:28.899519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:28.899519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:28.899519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:28.899519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:28.899519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:28.899519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:28.899519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:28.899519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:28.899519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:28.899519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:28.899519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:28:28.899519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:28:28.899519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:28:28.899519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:28:28.900763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:28:28.900763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:28:28.900763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:28:28.900763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:28:28.900763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:28.900763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:28.900763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:28.900763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:28.900763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:28.900763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:28.900763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:28.900763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:28:28.900763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:28.900763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:28.900763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:28.900763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:28.907358 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:28.907358 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:28.907358 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:28.907358 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:28.907358 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:28:28.907358 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:28:28.907358 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:28:28.907358 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:28:28.907358 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:28:28.907358 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:28:28.907358 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 31 14:28:28.907358 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:28:28.907358 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 584/1208] 10.222.0.67 () {56 vars in 1102 bytes} [Tue May 31 14:28:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:28.909499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a9158-dfad-458a-8e56-1a5d2a4f2722 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:28.967894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:29.043082 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a9158-dfad-458a-8e56-1a5d2a4f2722 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:29.082194 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c405a197-b593-407e-b3bd-0410660c4d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:29.083524 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405a197-b593-407e-b3bd-0410660c4d15 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:29.083790 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405a197-b593-407e-b3bd-0410660c4d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:29.084122 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405a197-b593-407e-b3bd-0410660c4d15 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:29.085015 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c405a197-b593-407e-b3bd-0410660c4d15 None admin] RBAC: Adding query filter params (name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:29.085361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c405a197-b593-407e-b3bd-0410660c4d15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:29.086801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c405a197-b593-407e-b3bd-0410660c4d15 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:29.096205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 625/1209] 10.222.0.67 () {56 vars in 1103 bytes} [Tue May 31 14:28:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:29.102527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301340dc-a9e5-4d22-862c-885e464ff9cb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:29.168535 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:29.229874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301340dc-a9e5-4d22-862c-885e464ff9cb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:29.272994 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5d747-8e7a-40b0-a90e-700b496f7fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:29.273820 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5d747-8e7a-40b0-a90e-700b496f7fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:29.274109 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5d747-8e7a-40b0-a90e-700b496f7fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:29.274391 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5d747-8e7a-40b0-a90e-700b496f7fa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:29.286938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5d747-8e7a-40b0-a90e-700b496f7fa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:29.288438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5d747-8e7a-40b0-a90e-700b496f7fa3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:29.309318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-70d5d747-8e7a-40b0-a90e-700b496f7fa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:29.309318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:29.309318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:29.309318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:29.309318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:29.309318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:29.309318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:29.309318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:29.309318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:29.309318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:29.309318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:29.309318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:29.309318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:28:29.309318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:28:29.309318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:28:29.310373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:28:29.310373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:29.310373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:29.310373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:29.310373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:29.310373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:29.310373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:29.310373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:29.310373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:28:29.310373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:29.310373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:29.310373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:29.310373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:29.310373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:29.310373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:29.310373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:29.310373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:29.311370 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:28:29.311370 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:28:29.311370 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:29.311370 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:28:29.312500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 585/1210] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:28:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:29.318675 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c405a197-b593-407e-b3bd-0410660c4d15 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:29.460040 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:29.463986 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c405a197-b593-407e-b3bd-0410660c4d15 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:29.523612 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4079e-8414-488b-99a3-0a4cc0ef6a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:29.525045 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4079e-8414-488b-99a3-0a4cc0ef6a90 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:29.525045 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4079e-8414-488b-99a3-0a4cc0ef6a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:29.525239 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4079e-8414-488b-99a3-0a4cc0ef6a90 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:29.525847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4079e-8414-488b-99a3-0a4cc0ef6a90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:29.526290 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4079e-8414-488b-99a3-0a4cc0ef6a90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:29.528020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4079e-8414-488b-99a3-0a4cc0ef6a90 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:29.546511 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 626/1211] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:28:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:28:29.553052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5d747-8e7a-40b0-a90e-700b496f7fa3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:29.677525 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5d747-8e7a-40b0-a90e-700b496f7fa3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:29.679483 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:29.722590 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb817945-4fba-4043-bc79-5edfe39e8739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:29.723394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb817945-4fba-4043-bc79-5edfe39e8739 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:29.723720 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb817945-4fba-4043-bc79-5edfe39e8739 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:29.724002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb817945-4fba-4043-bc79-5edfe39e8739 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:29.734142 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb817945-4fba-4043-bc79-5edfe39e8739 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:29.736777 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb817945-4fba-4043-bc79-5edfe39e8739 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:29.752709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-bb817945-4fba-4043-bc79-5edfe39e8739 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 31 14:28:29.752709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:29.752709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:29.752709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:29.752709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:29.752709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:29.752709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:29.752709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:29.752709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:29.752709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:29.752709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:29.752709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:29.752709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:28:29.752709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:28:29.752709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:28:29.752709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:28:29.753937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:29.753937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:29.753937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:28:29.753937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:28:29.753937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:28:29.753937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:28:29.753937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:29.753937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:29.753937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:29.753937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:29.753937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:29.753937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:28:29.753937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:29.753937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:29.753937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:29.753937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:29.753937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:29.753937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:29.755772 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:29.755772 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:29.755772 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:28:29.755772 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:28:29.755772 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:28:29.755772 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:28:29.755772 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:28:29.755772 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:28:29.755772 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 31 14:28:29.755772 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:28:29.755772 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 586/1212] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:28:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:29.761693 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4079e-8414-488b-99a3-0a4cc0ef6a90 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:29.891918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4079e-8414-488b-99a3-0a4cc0ef6a90 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:29.946786 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5afd2-49d8-474e-8813-0b9e3d37eb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:29.947925 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5afd2-49d8-474e-8813-0b9e3d37eb35 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:29.948353 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5afd2-49d8-474e-8813-0b9e3d37eb35 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:29.948638 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5afd2-49d8-474e-8813-0b9e3d37eb35 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:29.949551 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5afd2-49d8-474e-8813-0b9e3d37eb35 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:29.949873 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5afd2-49d8-474e-8813-0b9e3d37eb35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:29.951646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5afd2-49d8-474e-8813-0b9e3d37eb35 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:29.986771 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 627/1213] 10.222.0.67 () {56 vars in 1143 bytes} [Tue May 31 14:28:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:29.989401 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb817945-4fba-4043-bc79-5edfe39e8739 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:30.161201 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb817945-4fba-4043-bc79-5edfe39e8739 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:30.206952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048890ff-3dfd-47d5-8e7e-3de4e972ff38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:30.207733 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048890ff-3dfd-47d5-8e7e-3de4e972ff38 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:30.208067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048890ff-3dfd-47d5-8e7e-3de4e972ff38 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:30.208358 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048890ff-3dfd-47d5-8e7e-3de4e972ff38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:30.245189 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:30.293550 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048890ff-3dfd-47d5-8e7e-3de4e972ff38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:30.295371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048890ff-3dfd-47d5-8e7e-3de4e972ff38 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:30.408482 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 587/1214] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:28:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 31 14:28:30.417511 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5afd2-49d8-474e-8813-0b9e3d37eb35 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:30.446880 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:30.559340 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5afd2-49d8-474e-8813-0b9e3d37eb35 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:30.609244 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca63e7b-728a-4810-96f3-d7d530d7cb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:30.610364 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca63e7b-728a-4810-96f3-d7d530d7cb71 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:30.610924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca63e7b-728a-4810-96f3-d7d530d7cb71 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:30.611429 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca63e7b-728a-4810-96f3-d7d530d7cb71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:30.625260 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca63e7b-728a-4810-96f3-d7d530d7cb71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:30.626777 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca63e7b-728a-4810-96f3-d7d530d7cb71 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:30.645302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-1ca63e7b-728a-4810-96f3-d7d530d7cb71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:30.645302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:30.645302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:30.645302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:30.645302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:30.645302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:30.645302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:30.645302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:30.645302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:30.645302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:30.645302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:30.645302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:30.645302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:28:30.645302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:28:30.645302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:28:30.646273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:28:30.646273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:30.646273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:30.646273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:30.646273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:30.646273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:30.646273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:30.646273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:30.646273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:28:30.646273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:30.646273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:30.646273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:30.646273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:30.646273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:30.646273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:30.646273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:30.646273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:30.653079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:28:30.653079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:28:30.653079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:30.653079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:28:30.653079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 628/1215] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:28:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:28:30.653389 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048890ff-3dfd-47d5-8e7e-3de4e972ff38 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:30.739434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:30.795472 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048890ff-3dfd-47d5-8e7e-3de4e972ff38 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:30.858387 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91a6d8-28c4-486b-b214-aa01d1fa9f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:30.859104 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91a6d8-28c4-486b-b214-aa01d1fa9f5c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:30.860432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91a6d8-28c4-486b-b214-aa01d1fa9f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:30.860432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91a6d8-28c4-486b-b214-aa01d1fa9f5c None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:30.860432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c91a6d8-28c4-486b-b214-aa01d1fa9f5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:30.860836 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c91a6d8-28c4-486b-b214-aa01d1fa9f5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:30.862063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c91a6d8-28c4-486b-b214-aa01d1fa9f5c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:30.876243 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 588/1216] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:28:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:30.883986 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca63e7b-728a-4810-96f3-d7d530d7cb71 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:30.992346 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:31.047177 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca63e7b-728a-4810-96f3-d7d530d7cb71 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:31.102371 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e3efa-5904-446f-b7bb-e830d2dc06fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:31.103425 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e3efa-5904-446f-b7bb-e830d2dc06fa None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:31.103758 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e3efa-5904-446f-b7bb-e830d2dc06fa None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:31.104137 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e3efa-5904-446f-b7bb-e830d2dc06fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:31.116513 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e3efa-5904-446f-b7bb-e830d2dc06fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:31.119180 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e3efa-5904-446f-b7bb-e830d2dc06fa None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:31.142083 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-de0e3efa-5904-446f-b7bb-e830d2dc06fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:28:31.142083 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:31.142083 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:31.142083 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:31.142083 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:31.142083 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:31.142083 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:31.142083 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:31.142083 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:31.142083 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:31.142083 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:31.142083 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:31.142083 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:28:31.142083 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:28:31.142083 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:28:31.143109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:28:31.143109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:31.143109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:31.143109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:31.143109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:31.143109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:31.143109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:31.143109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:31.143109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:28:31.143109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:31.143109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:31.143109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:31.143109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:31.143109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:31.143109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:31.143109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:31.143109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:31.144030 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:28:31.144030 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:28:31.144030 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:28:31.144030 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:28:31.144030 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:28:31.144030 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:28:31.144030 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:28:31.144030 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:28:31.146983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 629/1217] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:28:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:31.151353 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91a6d8-28c4-486b-b214-aa01d1fa9f5c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:31.295798 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91a6d8-28c4-486b-b214-aa01d1fa9f5c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:31.325381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:31.339968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f4800-f1ed-43ea-a007-83ac2de1ca9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:31.340852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f4800-f1ed-43ea-a007-83ac2de1ca9d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:31.341278 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f4800-f1ed-43ea-a007-83ac2de1ca9d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:31.341695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f4800-f1ed-43ea-a007-83ac2de1ca9d None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:31.342769 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f4800-f1ed-43ea-a007-83ac2de1ca9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:31.343245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f4800-f1ed-43ea-a007-83ac2de1ca9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:31.345190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f4800-f1ed-43ea-a007-83ac2de1ca9d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:31.359357 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 589/1218] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:28:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:31.366033 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e3efa-5904-446f-b7bb-e830d2dc06fa None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:31.511588 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e3efa-5904-446f-b7bb-e830d2dc06fa None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:31.552208 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:31.559973 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d87d7-dc35-48b6-b7d4-c30faa07899b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:31.561001 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d87d7-dc35-48b6-b7d4-c30faa07899b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:31.561370 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d87d7-dc35-48b6-b7d4-c30faa07899b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:31.561771 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d87d7-dc35-48b6-b7d4-c30faa07899b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:31.562981 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d87d7-dc35-48b6-b7d4-c30faa07899b None admin] RBAC: Adding query filter params (role.id=5956aa478bfc4d03919395e637e61c83, scope.project.id=f965a93a2f674598890bd8cafededb2c, user.id=6c1c13515a8b41a781478dc65e6a88b5) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:31.563318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d87d7-dc35-48b6-b7d4-c30faa07899b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:31.564859 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d87d7-dc35-48b6-b7d4-c30faa07899b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:31.573870 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 630/1219] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:28:31 2022] GET /identity/v3/role_assignments?user.id=6c1c13515a8b41a781478dc65e6a88b5&scope.project.id=f965a93a2f674598890bd8cafededb2c&role.id=5956aa478bfc4d03919395e637e61c83 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:34.357580 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9d951-935c-45a8-b846-d08bc426ef91 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:34.358166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9d951-935c-45a8-b846-d08bc426ef91 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:34.358166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9d951-935c-45a8-b846-d08bc426ef91 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:34.359802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 590/1220] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:28:34.367232 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43b1eb-2ba3-4474-ba66-ba1a6ec0a402 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:34.368098 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43b1eb-2ba3-4474-ba66-ba1a6ec0a402 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:34.368098 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43b1eb-2ba3-4474-ba66-ba1a6ec0a402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:34.390636 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-7c43b1eb-2ba3-4474-ba66-ba1a6ec0a402 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:28:34.392621 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7c43b1eb-2ba3-4474-ba66-ba1a6ec0a402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:28:34.483491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 631/1221] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:28:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:28:34.508705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d9d951-935c-45a8-b846-d08bc426ef91 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:34.657048 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d9d951-935c-45a8-b846-d08bc426ef91 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:34.697676 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff5c6f-2b28-4f8f-86e3-cf8af9ac36cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:34.698492 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff5c6f-2b28-4f8f-86e3-cf8af9ac36cd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:34.698823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff5c6f-2b28-4f8f-86e3-cf8af9ac36cd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:34.699105 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff5c6f-2b28-4f8f-86e3-cf8af9ac36cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:34.712348 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff5c6f-2b28-4f8f-86e3-cf8af9ac36cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:28:34.724744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-6dff5c6f-2b28-4f8f-86e3-cf8af9ac36cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 31 14:28:34.724744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:34.724744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:34.724744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:34.724744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:34.724744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:34.724744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:34.724744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:34.724744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:34.724744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:34.724744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:34.724744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:34.724744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:28:34.724744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:28:34.724744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:28:34.724744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:28:34.725938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:28:34.725938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:28:34.725938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:28:34.725938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:28:34.725938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:34.725938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:34.725938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:34.725938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:34.725938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:34.725938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:34.725938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:34.725938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:28:34.725938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:34.725938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:34.725938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:34.725938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:34.731788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:34.731788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:34.731788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:34.731788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:34.731788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:28:34.731788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:28:34.731788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:28:34.731788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:28:34.731788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:28:34.731788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:28:34.731788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 31 14:28:34.731788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:28:34.731788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 591/1222] 10.222.0.67 () {56 vars in 1102 bytes} [Tue May 31 14:28:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:28:34.733230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43b1eb-2ba3-4474-ba66-ba1a6ec0a402 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:34.869858 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43b1eb-2ba3-4474-ba66-ba1a6ec0a402 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:34.908400 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:34.923304 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03e69f-d134-409e-85fc-989405ba93e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:34.924219 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03e69f-d134-409e-85fc-989405ba93e8 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:34.924614 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03e69f-d134-409e-85fc-989405ba93e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:34.924972 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03e69f-d134-409e-85fc-989405ba93e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:34.926622 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03e69f-d134-409e-85fc-989405ba93e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:34.926959 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03e69f-d134-409e-85fc-989405ba93e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:34.928636 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03e69f-d134-409e-85fc-989405ba93e8 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:34.939105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 632/1223] 10.222.0.67 () {56 vars in 1103 bytes} [Tue May 31 14:28:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:34.945661 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff5c6f-2b28-4f8f-86e3-cf8af9ac36cd None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:35.001091 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:35.079699 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff5c6f-2b28-4f8f-86e3-cf8af9ac36cd None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:35.123478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56d6fa-64f6-44d3-b26e-eeb82203a339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:35.124294 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56d6fa-64f6-44d3-b26e-eeb82203a339 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:35.124577 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56d6fa-64f6-44d3-b26e-eeb82203a339 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:35.124853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56d6fa-64f6-44d3-b26e-eeb82203a339 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:35.137795 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56d6fa-64f6-44d3-b26e-eeb82203a339 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:35.139986 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56d6fa-64f6-44d3-b26e-eeb82203a339 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:35.167281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-df56d6fa-64f6-44d3-b26e-eeb82203a339 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:35.167281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:35.167281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:35.167281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:35.167281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:35.167281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:35.167281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:35.167281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:35.167281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:35.167281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:35.167281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:35.167281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:35.167281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:28:35.167281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:28:35.167281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:28:35.168404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:28:35.168404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:35.168404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:35.168404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:35.168404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:35.168404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:35.168404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:35.168404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:35.168404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:28:35.168404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:35.168404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:35.168404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:35.168404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:35.168404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:35.168404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:35.168404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:35.168404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:35.169514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:28:35.169514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:28:35.169514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:35.169514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:28:35.169743 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 592/1224] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:28:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:28:35.178562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03e69f-d134-409e-85fc-989405ba93e8 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:35.310591 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03e69f-d134-409e-85fc-989405ba93e8 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:35.357046 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbe451-71df-4bd6-834e-0258e446fe52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:35.358018 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbe451-71df-4bd6-834e-0258e446fe52 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:35.358438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbe451-71df-4bd6-834e-0258e446fe52 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:35.358888 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbe451-71df-4bd6-834e-0258e446fe52 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:35.360008 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fbe451-71df-4bd6-834e-0258e446fe52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:35.360465 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fbe451-71df-4bd6-834e-0258e446fe52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:35.362576 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fbe451-71df-4bd6-834e-0258e446fe52 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:35.381489 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 633/1225] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:35.387889 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56d6fa-64f6-44d3-b26e-eeb82203a339 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:35.512153 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:35.521644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56d6fa-64f6-44d3-b26e-eeb82203a339 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:35.565322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b19837-6c11-4829-9349-c9d36aa53aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:35.566166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b19837-6c11-4829-9349-c9d36aa53aee None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:35.567027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b19837-6c11-4829-9349-c9d36aa53aee None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:35.567348 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b19837-6c11-4829-9349-c9d36aa53aee None admin] PATH_INFO: `/v3/users/placement` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:35.584292 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b19837-6c11-4829-9349-c9d36aa53aee None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:35.585913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b19837-6c11-4829-9349-c9d36aa53aee None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:35.607671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-54b19837-6c11-4829-9349-c9d36aa53aee None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 31 14:28:35.607671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:35.607671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:35.607671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:35.607671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:35.607671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:35.607671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:35.607671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:35.607671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:35.607671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:35.607671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:35.607671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:35.607671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:28:35.607671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:28:35.607671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:28:35.607671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:28:35.609427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:35.609427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:35.609427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:28:35.609427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:28:35.609427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:28:35.609427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:28:35.609427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:35.609427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:35.609427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:35.609427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:35.609427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:35.609427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:28:35.609427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:35.609427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:35.609427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:35.609427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:35.609427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:35.609427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:35.611486 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:35.611486 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:35.611486 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:28:35.611486 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:28:35.611486 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:28:35.611486 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:28:35.611486 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:28:35.611486 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:28:35.611486 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 31 14:28:35.611486 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:28:35.611486 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 593/1226] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:28:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:35.617762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbe451-71df-4bd6-834e-0258e446fe52 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:35.674551 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:35.746344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbe451-71df-4bd6-834e-0258e446fe52 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:35.792015 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763b25c-acb1-4daa-8a96-4e2dc0a951aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:35.793134 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763b25c-acb1-4daa-8a96-4e2dc0a951aa None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:35.793423 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763b25c-acb1-4daa-8a96-4e2dc0a951aa None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:35.793794 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763b25c-acb1-4daa-8a96-4e2dc0a951aa None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:35.794950 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c763b25c-acb1-4daa-8a96-4e2dc0a951aa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:35.795324 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c763b25c-acb1-4daa-8a96-4e2dc0a951aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:35.797605 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c763b25c-acb1-4daa-8a96-4e2dc0a951aa None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:35.823428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 634/1227] 10.222.0.67 () {56 vars in 1143 bytes} [Tue May 31 14:28:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:28:35.825342 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b19837-6c11-4829-9349-c9d36aa53aee None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:35.959138 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b19837-6c11-4829-9349-c9d36aa53aee None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:35.992804 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a10ee2-21c1-4d42-8234-3f58ee0a682e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:35.993601 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a10ee2-21c1-4d42-8234-3f58ee0a682e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:35.993884 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a10ee2-21c1-4d42-8234-3f58ee0a682e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:35.994158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a10ee2-21c1-4d42-8234-3f58ee0a682e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:36.008998 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a10ee2-21c1-4d42-8234-3f58ee0a682e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:36.010898 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a10ee2-21c1-4d42-8234-3f58ee0a682e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:36.032852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-88a10ee2-21c1-4d42-8234-3f58ee0a682e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:36.032852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:36.032852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:36.032852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:36.032852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:36.032852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:36.032852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:36.032852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:36.032852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:36.032852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:36.032852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:36.032852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:36.032852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:28:36.032852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:28:36.032852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:28:36.033980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:28:36.033980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:36.033980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:36.033980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:36.033980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:36.033980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:36.033980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:36.033980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:36.033980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:28:36.033980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:36.033980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:36.033980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:36.033980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:36.033980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:36.033980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:36.033980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:36.033980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:36.035255 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:28:36.035255 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:28:36.035255 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:36.035255 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:28:36.039112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 594/1228] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:28:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:28:36.040218 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763b25c-acb1-4daa-8a96-4e2dc0a951aa None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:36.186981 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763b25c-acb1-4daa-8a96-4e2dc0a951aa None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:36.189855 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:36.233675 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dd197-b2ec-4214-865f-221ca1ed63e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:36.234540 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dd197-b2ec-4214-865f-221ca1ed63e2 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:36.234949 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dd197-b2ec-4214-865f-221ca1ed63e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:36.235267 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dd197-b2ec-4214-865f-221ca1ed63e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:36.236343 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401dd197-b2ec-4214-865f-221ca1ed63e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:36.236813 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401dd197-b2ec-4214-865f-221ca1ed63e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:36.239298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401dd197-b2ec-4214-865f-221ca1ed63e2 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:36.254979 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 635/1229] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:36.260799 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a10ee2-21c1-4d42-8234-3f58ee0a682e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:36.271676 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:36.387917 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a10ee2-21c1-4d42-8234-3f58ee0a682e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:36.430673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d42e3a-5091-425b-a32c-77cff08acd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:36.431507 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d42e3a-5091-425b-a32c-77cff08acd07 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:36.432387 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d42e3a-5091-425b-a32c-77cff08acd07 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:36.432387 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d42e3a-5091-425b-a32c-77cff08acd07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:36.438673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d42e3a-5091-425b-a32c-77cff08acd07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:36.440081 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d42e3a-5091-425b-a32c-77cff08acd07 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:36.455434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-40d42e3a-5091-425b-a32c-77cff08acd07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:28:36.455434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:36.455434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:36.455434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:36.455434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:36.455434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:36.455434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:36.455434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:36.455434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:36.455434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:36.455434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:36.455434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:36.455434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:28:36.455434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:28:36.455434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:28:36.456440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:28:36.456440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:36.456440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:36.456440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:36.456440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:36.456440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:36.456440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:36.456440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:36.456440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:28:36.456440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:36.456440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:36.456440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:36.456440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:36.456440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:36.456440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:36.456440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:36.456440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:36.458009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:28:36.458009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:28:36.458009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:28:36.458009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:28:36.458009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:28:36.458009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:28:36.458009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:28:36.458009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:28:36.458009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 595/1230] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:28:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:36.464344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dd197-b2ec-4214-865f-221ca1ed63e2 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:36.586373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dd197-b2ec-4214-865f-221ca1ed63e2 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:36.623251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763296ea-9a74-4308-bb8e-331e657dee81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:36.624189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763296ea-9a74-4308-bb8e-331e657dee81 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:36.624464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763296ea-9a74-4308-bb8e-331e657dee81 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:36.624747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763296ea-9a74-4308-bb8e-331e657dee81 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:36.625538 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763296ea-9a74-4308-bb8e-331e657dee81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:36.625854 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763296ea-9a74-4308-bb8e-331e657dee81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:36.627334 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763296ea-9a74-4308-bb8e-331e657dee81 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:36.641572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 636/1231] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:28:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:28:36.649994 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d42e3a-5091-425b-a32c-77cff08acd07 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:36.733247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:36.777438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d42e3a-5091-425b-a32c-77cff08acd07 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:36.822896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2fa4c-36a6-47cd-b175-4944e1c7d908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:36.823797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2fa4c-36a6-47cd-b175-4944e1c7d908 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:36.824164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2fa4c-36a6-47cd-b175-4944e1c7d908 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:36.824445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2fa4c-36a6-47cd-b175-4944e1c7d908 None admin] PATH_INFO: `/v3/projects/f965a93a2f674598890bd8cafededb2c/users/6c1c13515a8b41a781478dc65e6a88b5/roles/5956aa478bfc4d03919395e637e61c83` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:36.846357 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a2fa4c-36a6-47cd-b175-4944e1c7d908 None admin] RBAC: Authorizing `identity:create_grant(project_id=f965a93a2f674598890bd8cafededb2c, user_id=6c1c13515a8b41a781478dc65e6a88b5, role_id=5956aa478bfc4d03919395e637e61c83)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:36.848718 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a2fa4c-36a6-47cd-b175-4944e1c7d908 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:36.865821 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 596/1232] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:28:36 2022] PUT /identity/v3/projects/f965a93a2f674598890bd8cafededb2c/users/6c1c13515a8b41a781478dc65e6a88b5/roles/5956aa478bfc4d03919395e637e61c83 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:28:39.426637 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdf133-6026-4ded-a7a3-31d4e9d950f4 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:39.427238 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdf133-6026-4ded-a7a3-31d4e9d950f4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:39.427238 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdf133-6026-4ded-a7a3-31d4e9d950f4 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:39.428684 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 637/1233] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:28:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:28:39.434599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0896c56-6fa6-46d3-9de6-4fc2350edf5b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:39.435144 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0896c56-6fa6-46d3-9de6-4fc2350edf5b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:39.435245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0896c56-6fa6-46d3-9de6-4fc2350edf5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:39.453554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f0896c56-6fa6-46d3-9de6-4fc2350edf5b None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:28:39.454861 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f0896c56-6fa6-46d3-9de6-4fc2350edf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:28:39.531444 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 597/1234] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:28:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:28:39.556774 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fdf133-6026-4ded-a7a3-31d4e9d950f4 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:39.583928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:39.677375 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fdf133-6026-4ded-a7a3-31d4e9d950f4 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:39.720205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1606f-2a8d-4e5c-a430-a1222ea76d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:39.721005 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1606f-2a8d-4e5c-a430-a1222ea76d8a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:39.721289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1606f-2a8d-4e5c-a430-a1222ea76d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:39.721583 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1606f-2a8d-4e5c-a430-a1222ea76d8a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:39.734733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb1606f-2a8d-4e5c-a430-a1222ea76d8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:28:39.743797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-beb1606f-2a8d-4e5c-a430-a1222ea76d8a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 31 14:28:39.743797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:39.743797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:39.743797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:39.743797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:39.743797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:39.743797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:39.743797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:39.743797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:39.743797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:39.743797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:39.743797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:39.743797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:28:39.743797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:28:39.743797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:28:39.743797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:28:39.745038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:28:39.745038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:28:39.745038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:28:39.745038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:28:39.745038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:39.745038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:39.745038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:39.745038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:39.745038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:39.745038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:39.745038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:39.745038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:28:39.745038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:39.745038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:39.745038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:39.745038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:39.745985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:39.745985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:39.745985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:39.745985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:39.745985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:28:39.745985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:28:39.745985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:28:39.745985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:28:39.745985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:28:39.745985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:28:39.745985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 31 14:28:39.745985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:28:39.745985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 638/1235] 10.222.0.67 () {56 vars in 1102 bytes} [Tue May 31 14:28:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:39.751465 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0896c56-6fa6-46d3-9de6-4fc2350edf5b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:39.803591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:39.865892 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0896c56-6fa6-46d3-9de6-4fc2350edf5b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:39.905946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3a235-71de-4f7a-b4b5-82f1d5d827cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:39.906664 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3a235-71de-4f7a-b4b5-82f1d5d827cc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:39.906945 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3a235-71de-4f7a-b4b5-82f1d5d827cc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:39.907221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3a235-71de-4f7a-b4b5-82f1d5d827cc None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:39.907973 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3a235-71de-4f7a-b4b5-82f1d5d827cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:39.908284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3a235-71de-4f7a-b4b5-82f1d5d827cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:39.909684 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3a235-71de-4f7a-b4b5-82f1d5d827cc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:39.918874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 598/1236] 10.222.0.67 () {56 vars in 1103 bytes} [Tue May 31 14:28:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:39.925113 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1606f-2a8d-4e5c-a430-a1222ea76d8a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:40.032389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:40.053419 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1606f-2a8d-4e5c-a430-a1222ea76d8a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:40.094169 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1195e6e-90b7-460b-b191-696e778e7554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:40.095153 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1195e6e-90b7-460b-b191-696e778e7554 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:40.095846 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1195e6e-90b7-460b-b191-696e778e7554 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:40.096278 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1195e6e-90b7-460b-b191-696e778e7554 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:40.109753 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1195e6e-90b7-460b-b191-696e778e7554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:40.111985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1195e6e-90b7-460b-b191-696e778e7554 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:40.131760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-b1195e6e-90b7-460b-b191-696e778e7554 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:40.131760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:40.131760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:40.131760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:40.131760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:40.131760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:40.131760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:40.131760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:40.131760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:40.131760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:40.131760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:40.131760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:40.131760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:28:40.131760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:28:40.131760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:28:40.132951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:28:40.132951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:40.132951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:40.132951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:40.132951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:40.132951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:40.132951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:40.132951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:40.132951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:28:40.132951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:40.132951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:40.132951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:40.132951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:40.132951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:40.132951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:40.132951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:40.132951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:40.133921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:28:40.133921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:28:40.133921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:40.133921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:28:40.133921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 639/1237] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:28:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:28:40.139510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3a235-71de-4f7a-b4b5-82f1d5d827cc None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:40.264936 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3a235-71de-4f7a-b4b5-82f1d5d827cc None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:40.281784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:40.321352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f0000-1a7f-4bde-b5b0-7721c74b67a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:40.322335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f0000-1a7f-4bde-b5b0-7721c74b67a1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:40.322734 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f0000-1a7f-4bde-b5b0-7721c74b67a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:40.323024 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f0000-1a7f-4bde-b5b0-7721c74b67a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:40.323957 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f0000-1a7f-4bde-b5b0-7721c74b67a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:40.324452 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f0000-1a7f-4bde-b5b0-7721c74b67a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:40.325828 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f0000-1a7f-4bde-b5b0-7721c74b67a1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:40.343191 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 599/1238] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:40.349391 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1195e6e-90b7-460b-b191-696e778e7554 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:40.463312 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1195e6e-90b7-460b-b191-696e778e7554 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:40.500770 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4a53c-1e20-4d2c-9661-f55e5d4b8eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:40.501579 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4a53c-1e20-4d2c-9661-f55e5d4b8eab None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:40.501860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4a53c-1e20-4d2c-9661-f55e5d4b8eab None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:40.502133 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4a53c-1e20-4d2c-9661-f55e5d4b8eab None admin] PATH_INFO: `/v3/users/placement` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:40.505412 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:40.510743 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4a53c-1e20-4d2c-9661-f55e5d4b8eab None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:40.512323 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4a53c-1e20-4d2c-9661-f55e5d4b8eab None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:40.534563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-2fe4a53c-1e20-4d2c-9661-f55e5d4b8eab None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 31 14:28:40.534563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:40.534563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:40.534563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:40.534563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:40.534563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:40.534563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:40.534563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:40.534563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:40.534563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:40.534563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:40.534563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:40.534563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:28:40.534563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:28:40.534563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:28:40.534563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:28:40.536317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:40.536317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:40.536317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:28:40.536317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:28:40.536317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:28:40.536317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:28:40.536317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:40.536317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:40.536317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:40.536317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:40.536317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:40.536317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:28:40.536317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:40.536317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:40.536317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:40.536317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:40.536317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:40.536317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:40.537313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:40.537313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:40.537313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:28:40.537313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:28:40.537313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:28:40.537313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:28:40.537313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:28:40.537313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:28:40.537313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 31 14:28:40.537313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:28:40.537313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 640/1239] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:28:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:40.545578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f0000-1a7f-4bde-b5b0-7721c74b67a1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:40.664468 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f0000-1a7f-4bde-b5b0-7721c74b67a1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:40.703035 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6fdfd-5270-4866-8172-4e7be6dd7b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:40.703833 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6fdfd-5270-4866-8172-4e7be6dd7b9d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:40.704237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6fdfd-5270-4866-8172-4e7be6dd7b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:40.704646 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6fdfd-5270-4866-8172-4e7be6dd7b9d None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:40.705537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6fdfd-5270-4866-8172-4e7be6dd7b9d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:40.705875 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6fdfd-5270-4866-8172-4e7be6dd7b9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:40.708154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6fdfd-5270-4866-8172-4e7be6dd7b9d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:40.727934 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 600/1240] 10.222.0.67 () {56 vars in 1143 bytes} [Tue May 31 14:28:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:40.733822 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4a53c-1e20-4d2c-9661-f55e5d4b8eab None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:40.853095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4a53c-1e20-4d2c-9661-f55e5d4b8eab None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:40.891946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1cc3a-6b62-4be9-be8f-e6df59375b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:40.892707 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1cc3a-6b62-4be9-be8f-e6df59375b1c None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:40.893016 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1cc3a-6b62-4be9-be8f-e6df59375b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:40.893298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1cc3a-6b62-4be9-be8f-e6df59375b1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:40.972992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:40.982689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1cc3a-6b62-4be9-be8f-e6df59375b1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:40.983909 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1cc3a-6b62-4be9-be8f-e6df59375b1c None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:41.097028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 641/1241] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:28:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 31 14:28:41.098956 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6fdfd-5270-4866-8172-4e7be6dd7b9d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:41.128177 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:41.206784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6fdfd-5270-4866-8172-4e7be6dd7b9d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:41.243738 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209809c-1a61-4738-a22a-412e4e9ab6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:41.244465 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209809c-1a61-4738-a22a-412e4e9ab6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:41.244752 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209809c-1a61-4738-a22a-412e4e9ab6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:41.245029 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209809c-1a61-4738-a22a-412e4e9ab6a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:41.255754 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f209809c-1a61-4738-a22a-412e4e9ab6a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:41.257218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f209809c-1a61-4738-a22a-412e4e9ab6a7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:41.275977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f209809c-1a61-4738-a22a-412e4e9ab6a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:41.275977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:41.275977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:41.275977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:41.275977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:41.275977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:41.275977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:41.275977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:41.275977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:41.275977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:41.275977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:41.275977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:41.275977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:28:41.275977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:28:41.275977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:28:41.277380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:28:41.277380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:41.277380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:41.277380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:41.277380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:41.277380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:41.277380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:41.277380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:41.277380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:28:41.277380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:41.277380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:41.277380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:41.277380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:41.277380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:41.277380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:41.277380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:41.277380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:41.278514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:28:41.278514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:28:41.278514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:41.278514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:28:41.278514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 601/1242] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:28:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:41.283375 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1cc3a-6b62-4be9-be8f-e6df59375b1c None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:41.406180 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:41.413638 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1cc3a-6b62-4be9-be8f-e6df59375b1c None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:41.451133 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abc1dd-69ba-4e8a-9d91-5ff72cd5c7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:41.451805 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abc1dd-69ba-4e8a-9d91-5ff72cd5c7be None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:41.452096 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abc1dd-69ba-4e8a-9d91-5ff72cd5c7be None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:41.452406 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abc1dd-69ba-4e8a-9d91-5ff72cd5c7be None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:41.453168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0abc1dd-69ba-4e8a-9d91-5ff72cd5c7be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:41.453481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0abc1dd-69ba-4e8a-9d91-5ff72cd5c7be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:41.454882 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0abc1dd-69ba-4e8a-9d91-5ff72cd5c7be None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:41.470897 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 642/1243] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:41.476755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209809c-1a61-4738-a22a-412e4e9ab6a7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:41.570627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:41.596259 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209809c-1a61-4738-a22a-412e4e9ab6a7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:41.640960 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbbcd0-105b-4c74-b9aa-d1a609122b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:41.642046 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbbcd0-105b-4c74-b9aa-d1a609122b7f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:41.642566 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbbcd0-105b-4c74-b9aa-d1a609122b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:41.642783 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbbcd0-105b-4c74-b9aa-d1a609122b7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:41.652041 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dbbcd0-105b-4c74-b9aa-d1a609122b7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:41.653637 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dbbcd0-105b-4c74-b9aa-d1a609122b7f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:41.670578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-d0dbbcd0-105b-4c74-b9aa-d1a609122b7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:28:41.670578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:41.670578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:41.670578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:41.670578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:41.670578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:41.670578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:41.670578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:41.670578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:41.670578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:41.670578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:41.670578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:41.670578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:28:41.670578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:28:41.670578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:28:41.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:28:41.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:41.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:41.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:41.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:41.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:41.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:41.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:41.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:28:41.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:41.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:41.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:41.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:41.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:41.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:41.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:41.672809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:41.674913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:28:41.674913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:28:41.674913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:28:41.674913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:28:41.674913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:28:41.674913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:28:41.674913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:28:41.674913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:28:41.674913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 602/1244] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:28:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:41.679898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abc1dd-69ba-4e8a-9d91-5ff72cd5c7be None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:41.808729 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abc1dd-69ba-4e8a-9d91-5ff72cd5c7be None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:41.848777 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1afc77-fd85-4639-a364-3eef26ed84ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:41.849414 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1afc77-fd85-4639-a364-3eef26ed84ad None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:41.849692 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1afc77-fd85-4639-a364-3eef26ed84ad None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:41.849983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1afc77-fd85-4639-a364-3eef26ed84ad None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:41.850933 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1afc77-fd85-4639-a364-3eef26ed84ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:41.851239 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1afc77-fd85-4639-a364-3eef26ed84ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:41.852768 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1afc77-fd85-4639-a364-3eef26ed84ad None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:41.868181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 643/1245] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:28:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:28:41.875164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbbcd0-105b-4c74-b9aa-d1a609122b7f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:42.005778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbbcd0-105b-4c74-b9aa-d1a609122b7f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:42.045980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44fee72-5c76-48a0-90e7-93ef9463d369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:42.046753 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fee72-5c76-48a0-90e7-93ef9463d369 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:42.047101 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fee72-5c76-48a0-90e7-93ef9463d369 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:42.047420 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fee72-5c76-48a0-90e7-93ef9463d369 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:42.048362 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44fee72-5c76-48a0-90e7-93ef9463d369 None admin] RBAC: Adding query filter params (role.id=5956aa478bfc4d03919395e637e61c83, scope.project.id=f965a93a2f674598890bd8cafededb2c, user.id=6c1c13515a8b41a781478dc65e6a88b5) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:42.048678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44fee72-5c76-48a0-90e7-93ef9463d369 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:42.050166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44fee72-5c76-48a0-90e7-93ef9463d369 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:42.066917 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 603/1246] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:28:41 2022] GET /identity/v3/role_assignments?user.id=6c1c13515a8b41a781478dc65e6a88b5&scope.project.id=f965a93a2f674598890bd8cafededb2c&role.id=5956aa478bfc4d03919395e637e61c83 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:44.753004 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf622d88-03e2-4bbb-8724-d823f038c738 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:44.754187 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf622d88-03e2-4bbb-8724-d823f038c738 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:44.754321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf622d88-03e2-4bbb-8724-d823f038c738 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:44.757373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 644/1247] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:28:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:28:44.765806 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77344b56-3db5-4b95-a626-6ce6eb3ff7a1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:44.766305 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77344b56-3db5-4b95-a626-6ce6eb3ff7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:44.766635 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77344b56-3db5-4b95-a626-6ce6eb3ff7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:44.773741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:44.788385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-77344b56-3db5-4b95-a626-6ce6eb3ff7a1 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:28:44.790309 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-77344b56-3db5-4b95-a626-6ce6eb3ff7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:28:44.890682 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 604/1248] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:28:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:28:44.917974 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf622d88-03e2-4bbb-8724-d823f038c738 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:44.944823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:45.089907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf622d88-03e2-4bbb-8724-d823f038c738 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:45.135466 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e7397-7892-4b24-abd8-d570fc4c8c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:45.136792 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e7397-7892-4b24-abd8-d570fc4c8c06 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:45.137293 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e7397-7892-4b24-abd8-d570fc4c8c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:45.137785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e7397-7892-4b24-abd8-d570fc4c8c06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:45.151009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e7397-7892-4b24-abd8-d570fc4c8c06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:28:45.160384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-5e6e7397-7892-4b24-abd8-d570fc4c8c06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:28:45.160384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:45.160384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:45.160384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:45.160384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:45.160384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:45.160384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:45.160384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:45.160384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:45.160384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:45.160384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:45.160384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:45.160384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:28:45.160384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:28:45.160384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:28:45.160384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:28:45.162189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:28:45.162189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:28:45.162189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:28:45.162189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:28:45.162189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:45.162189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:45.162189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:45.162189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:45.162189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:45.162189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:45.162189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:45.162189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:28:45.162189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:45.162189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:45.162189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:45.162189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:45.163568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:45.163568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:45.163568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:45.163568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:45.163568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:28:45.163568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:28:45.163568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:28:45.163568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:28:45.163568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:28:45.163568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:28:45.163568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:28:45.163568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:28:45.163568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 645/1249] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:28:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 31 14:28:45.174069 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77344b56-3db5-4b95-a626-6ce6eb3ff7a1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:45.300169 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77344b56-3db5-4b95-a626-6ce6eb3ff7a1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:45.334596 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:45.342459 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d85195-e70a-4af0-827b-b77ff89443f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:45.343327 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d85195-e70a-4af0-827b-b77ff89443f2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:45.343663 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d85195-e70a-4af0-827b-b77ff89443f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:45.344062 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d85195-e70a-4af0-827b-b77ff89443f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:45.345008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d85195-e70a-4af0-827b-b77ff89443f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:45.345474 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d85195-e70a-4af0-827b-b77ff89443f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:45.346950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d85195-e70a-4af0-827b-b77ff89443f2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:45.357680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 605/1250] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:28:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:45.366168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e7397-7892-4b24-abd8-d570fc4c8c06 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:45.471031 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:45.495448 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e7397-7892-4b24-abd8-d570fc4c8c06 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:45.529367 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634067a-6aee-4a18-b428-845d41abc7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:45.530242 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634067a-6aee-4a18-b428-845d41abc7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:45.530524 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634067a-6aee-4a18-b428-845d41abc7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:45.530873 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634067a-6aee-4a18-b428-845d41abc7c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:45.542429 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634067a-6aee-4a18-b428-845d41abc7c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:45.543926 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634067a-6aee-4a18-b428-845d41abc7c3 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:45.563613 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-c634067a-6aee-4a18-b428-845d41abc7c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:45.563613 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:45.563613 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:45.563613 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:45.563613 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:45.563613 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:45.563613 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:45.563613 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:45.563613 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:45.563613 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:45.563613 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:45.563613 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:45.563613 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:28:45.563613 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:28:45.563613 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:28:45.565550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:28:45.565550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:45.565550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:45.565550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:45.565550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:45.565550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:45.565550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:45.565550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:45.565550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:28:45.565550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:45.565550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:45.565550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:45.565550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:45.565550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:45.565550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:45.565550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:45.565550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:45.568027 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:28:45.568027 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:28:45.568027 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:45.568027 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:28:45.568027 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 646/1251] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:28:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:45.576453 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d85195-e70a-4af0-827b-b77ff89443f2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:45.711709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d85195-e70a-4af0-827b-b77ff89443f2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:45.764602 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe93801-793a-4157-a28b-831f0531cf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:45.765461 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe93801-793a-4157-a28b-831f0531cf2f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:45.765838 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe93801-793a-4157-a28b-831f0531cf2f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:45.766825 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe93801-793a-4157-a28b-831f0531cf2f None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:45.767453 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe93801-793a-4157-a28b-831f0531cf2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:45.767870 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe93801-793a-4157-a28b-831f0531cf2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:45.770051 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe93801-793a-4157-a28b-831f0531cf2f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:45.789632 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 606/1252] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:45.796210 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634067a-6aee-4a18-b428-845d41abc7c3 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:45.958828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634067a-6aee-4a18-b428-845d41abc7c3 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:46.006697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:46.021903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233cfa9-5931-4aa5-9e61-4e32c5f99e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:46.022968 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233cfa9-5931-4aa5-9e61-4e32c5f99e2a None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:46.023170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233cfa9-5931-4aa5-9e61-4e32c5f99e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:46.023456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233cfa9-5931-4aa5-9e61-4e32c5f99e2a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:46.034373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233cfa9-5931-4aa5-9e61-4e32c5f99e2a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:46.037294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233cfa9-5931-4aa5-9e61-4e32c5f99e2a None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:46.062396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-c233cfa9-5931-4aa5-9e61-4e32c5f99e2a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 31 14:28:46.062396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:46.062396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:46.062396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:46.062396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:46.062396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:46.062396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:46.062396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:46.062396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:46.062396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:46.062396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:46.062396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:46.062396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:28:46.062396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:28:46.062396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:28:46.062396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:28:46.065265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:46.065265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:46.065265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:28:46.065265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:28:46.065265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:28:46.065265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:28:46.065265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:46.065265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:46.065265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:46.065265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:46.065265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:46.065265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:28:46.065265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:46.065265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:46.065265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:46.065265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:46.065265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:46.065265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:46.067481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:46.067481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:46.067481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:28:46.067481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:28:46.067481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:28:46.067481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:28:46.067481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:28:46.067481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:28:46.067481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 31 14:28:46.067481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:28:46.067481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 647/1253] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:28:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:46.072722 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe93801-793a-4157-a28b-831f0531cf2f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:46.155877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:46.217504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe93801-793a-4157-a28b-831f0531cf2f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:46.274130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6ade0-edf2-45e3-b9e7-7136acc312c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:46.274893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6ade0-edf2-45e3-b9e7-7136acc312c6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:46.275348 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6ade0-edf2-45e3-b9e7-7136acc312c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:46.275929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6ade0-edf2-45e3-b9e7-7136acc312c6 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:46.276908 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc6ade0-edf2-45e3-b9e7-7136acc312c6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:46.277383 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc6ade0-edf2-45e3-b9e7-7136acc312c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:46.279482 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc6ade0-edf2-45e3-b9e7-7136acc312c6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:46.304534 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 607/1254] 10.222.0.67 () {56 vars in 1143 bytes} [Tue May 31 14:28:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:46.312186 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233cfa9-5931-4aa5-9e61-4e32c5f99e2a None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:46.450726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233cfa9-5931-4aa5-9e61-4e32c5f99e2a None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:46.491030 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833927a-309b-4faf-b74e-1d7e36d075a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:46.491855 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833927a-309b-4faf-b74e-1d7e36d075a2 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:46.492287 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833927a-309b-4faf-b74e-1d7e36d075a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:46.492570 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833927a-309b-4faf-b74e-1d7e36d075a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:46.556430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:46.570750 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b833927a-309b-4faf-b74e-1d7e36d075a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:46.572530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b833927a-309b-4faf-b74e-1d7e36d075a2 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:46.675064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 648/1255] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:28:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 31 14:28:46.681323 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6ade0-edf2-45e3-b9e7-7136acc312c6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:46.787796 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6ade0-edf2-45e3-b9e7-7136acc312c6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:46.802478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:46.825452 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b0012-3fed-48fc-9653-502a7ccc0fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:46.826174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b0012-3fed-48fc-9653-502a7ccc0fed None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:46.826456 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b0012-3fed-48fc-9653-502a7ccc0fed None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:46.826792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b0012-3fed-48fc-9653-502a7ccc0fed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:46.837842 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b0012-3fed-48fc-9653-502a7ccc0fed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:46.839342 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b0012-3fed-48fc-9653-502a7ccc0fed None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:46.858060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-4f4b0012-3fed-48fc-9653-502a7ccc0fed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:46.858060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:46.858060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:46.858060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:46.858060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:46.858060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:46.858060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:46.858060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:46.858060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:46.858060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:46.858060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:46.858060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:46.858060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:28:46.858060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:28:46.858060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:28:46.859245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:28:46.859245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:46.859245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:46.859245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:46.859245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:46.859245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:46.859245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:46.859245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:46.859245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:28:46.859245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:46.859245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:46.859245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:46.859245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:46.859245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:46.859245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:46.859245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:46.859245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:46.860309 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:28:46.860309 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:28:46.860309 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:46.860309 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:28:46.860309 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 608/1256] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:28:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:46.866043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833927a-309b-4faf-b74e-1d7e36d075a2 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:46.967846 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:46.983240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833927a-309b-4faf-b74e-1d7e36d075a2 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:47.018401 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c351fa-9826-41c0-8ea6-72a7d3753eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:47.019175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c351fa-9826-41c0-8ea6-72a7d3753eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:47.019461 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c351fa-9826-41c0-8ea6-72a7d3753eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:47.019746 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c351fa-9826-41c0-8ea6-72a7d3753eb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:47.020475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c351fa-9826-41c0-8ea6-72a7d3753eb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:47.020787 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c351fa-9826-41c0-8ea6-72a7d3753eb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:47.022170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c351fa-9826-41c0-8ea6-72a7d3753eb8 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:47.033881 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 649/1257] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:47.040236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b0012-3fed-48fc-9653-502a7ccc0fed None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:47.147147 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b0012-3fed-48fc-9653-502a7ccc0fed None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:47.184318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc2249-c5af-483e-acfe-490bc4902900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:47.185280 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc2249-c5af-483e-acfe-490bc4902900 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:47.185567 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc2249-c5af-483e-acfe-490bc4902900 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:47.185844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc2249-c5af-483e-acfe-490bc4902900 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:47.192831 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc2249-c5af-483e-acfe-490bc4902900 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:47.194275 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc2249-c5af-483e-acfe-490bc4902900 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:47.208864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-d1cc2249-c5af-483e-acfe-490bc4902900 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:28:47.208864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:47.208864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:47.208864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:47.208864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:47.208864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:47.208864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:47.208864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:47.208864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:47.208864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:47.208864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:47.208864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:47.208864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:28:47.208864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:28:47.208864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:28:47.209839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:28:47.209839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:47.209839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:47.209839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:47.209839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:47.209839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:47.209839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:47.209839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:47.209839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:28:47.209839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:47.209839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:47.209839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:47.209839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:47.209839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:47.209839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:47.209839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:47.209839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:47.210896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:28:47.210896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:28:47.210896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:28:47.210896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:28:47.210896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:28:47.210896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:28:47.210896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:28:47.210896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:28:47.210896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 609/1258] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:28:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:47.215967 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c351fa-9826-41c0-8ea6-72a7d3753eb8 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:47.316648 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c351fa-9826-41c0-8ea6-72a7d3753eb8 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:47.349869 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731ea60-bf15-4495-9987-195e15c333b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:47.350528 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731ea60-bf15-4495-9987-195e15c333b1 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:47.350849 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731ea60-bf15-4495-9987-195e15c333b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:47.351132 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731ea60-bf15-4495-9987-195e15c333b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:47.351920 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c731ea60-bf15-4495-9987-195e15c333b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:47.352235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c731ea60-bf15-4495-9987-195e15c333b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:47.353667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c731ea60-bf15-4495-9987-195e15c333b1 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:47.365467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 650/1259] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:28:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:47.371570 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc2249-c5af-483e-acfe-490bc4902900 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:47.397147 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:47.469019 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc2249-c5af-483e-acfe-490bc4902900 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:47.501689 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf2bec-716a-41b1-819b-7fb65140e8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:47.502341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf2bec-716a-41b1-819b-7fb65140e8bc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:47.502628 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf2bec-716a-41b1-819b-7fb65140e8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:47.502939 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf2bec-716a-41b1-819b-7fb65140e8bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:47.504274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf2bec-716a-41b1-819b-7fb65140e8bc None admin] RBAC: Adding query filter params (role.id=e8d679596ffb45939229d1e2a13ce9b0, scope.project.id=f965a93a2f674598890bd8cafededb2c, user.id=6c1c13515a8b41a781478dc65e6a88b5) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:47.504588 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf2bec-716a-41b1-819b-7fb65140e8bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:47.506005 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf2bec-716a-41b1-819b-7fb65140e8bc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:47.511392 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 610/1260] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:28:47 2022] GET /identity/v3/role_assignments?user.id=6c1c13515a8b41a781478dc65e6a88b5&scope.project.id=f965a93a2f674598890bd8cafededb2c&role.id=e8d679596ffb45939229d1e2a13ce9b0 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:49.776647 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce8087-b476-418b-9742-468d4a6390af None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:49.777202 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce8087-b476-418b-9742-468d4a6390af None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:49.777202 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce8087-b476-418b-9742-468d4a6390af None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:49.778560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 651/1261] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:28:49.783609 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f8faa-5a4e-472e-8a5f-f8fef912dde4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:49.783980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f8faa-5a4e-472e-8a5f-f8fef912dde4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:49.784177 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f8faa-5a4e-472e-8a5f-f8fef912dde4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:49.800678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-316f8faa-5a4e-472e-8a5f-f8fef912dde4 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:28:49.801864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-316f8faa-5a4e-472e-8a5f-f8fef912dde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:28:49.865948 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 611/1262] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:28:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:28:49.888404 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce8087-b476-418b-9742-468d4a6390af None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:49.899563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:49.998634 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce8087-b476-418b-9742-468d4a6390af None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:50.032343 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f13379-cd09-4631-8e88-0858871a251b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:50.033063 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f13379-cd09-4631-8e88-0858871a251b None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:50.033343 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f13379-cd09-4631-8e88-0858871a251b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:50.033617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f13379-cd09-4631-8e88-0858871a251b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:50.044026 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f13379-cd09-4631-8e88-0858871a251b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:28:50.052183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-63f13379-cd09-4631-8e88-0858871a251b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:28:50.052183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:50.052183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:50.052183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:50.052183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:50.052183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:50.052183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:50.052183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:50.052183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:50.052183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:50.052183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:50.052183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:50.052183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:28:50.052183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:28:50.052183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:28:50.052183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:28:50.053382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:28:50.053382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:28:50.053382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:28:50.053382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:28:50.053382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:50.053382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:50.053382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:50.053382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:50.053382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:50.053382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:50.053382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:50.053382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:28:50.053382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:50.053382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:50.053382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:50.053382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:50.054441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:50.054441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:50.054441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:50.054441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:50.054441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:28:50.054441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:28:50.054441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:28:50.054441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:28:50.054441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:28:50.054441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:28:50.054441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:28:50.054441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:28:50.054441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 652/1263] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:28:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:50.059437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f8faa-5a4e-472e-8a5f-f8fef912dde4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:50.077814 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:50.155693 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f8faa-5a4e-472e-8a5f-f8fef912dde4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:50.188800 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a030e6c-7b4c-485a-bf61-8a42fd205f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:50.189462 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a030e6c-7b4c-485a-bf61-8a42fd205f7c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:50.189744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a030e6c-7b4c-485a-bf61-8a42fd205f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:50.190018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a030e6c-7b4c-485a-bf61-8a42fd205f7c None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:50.190790 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a030e6c-7b4c-485a-bf61-8a42fd205f7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:50.191638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a030e6c-7b4c-485a-bf61-8a42fd205f7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:50.193009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a030e6c-7b4c-485a-bf61-8a42fd205f7c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:50.201088 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 612/1264] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:28:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:50.207345 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f13379-cd09-4631-8e88-0858871a251b None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:50.271863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:50.304974 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f13379-cd09-4631-8e88-0858871a251b None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:50.337124 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb118ce1-659d-4af2-bf01-bdd758b5d0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:50.337903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb118ce1-659d-4af2-bf01-bdd758b5d0d9 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:50.338189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb118ce1-659d-4af2-bf01-bdd758b5d0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:50.338466 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb118ce1-659d-4af2-bf01-bdd758b5d0d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:50.349176 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb118ce1-659d-4af2-bf01-bdd758b5d0d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:50.350622 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb118ce1-659d-4af2-bf01-bdd758b5d0d9 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:50.367603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-fb118ce1-659d-4af2-bf01-bdd758b5d0d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:50.367603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:50.367603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:50.367603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:50.367603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:50.367603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:50.367603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:50.367603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:50.367603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:50.367603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:50.367603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:50.367603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:50.367603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:28:50.367603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:28:50.367603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:28:50.368665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:28:50.368665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:50.368665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:50.368665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:50.368665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:50.368665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:50.368665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:50.368665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:50.368665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:28:50.368665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:50.368665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:50.368665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:50.368665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:50.368665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:50.368665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:50.368665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:50.368665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:50.369672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:28:50.369672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:28:50.369672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:50.369672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:28:50.369672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 653/1265] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:28:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:50.374570 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a030e6c-7b4c-485a-bf61-8a42fd205f7c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:50.453157 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:50.473110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a030e6c-7b4c-485a-bf61-8a42fd205f7c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:50.505006 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf162e-6845-4e3e-bcd3-c79580f38695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:50.505661 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf162e-6845-4e3e-bcd3-c79580f38695 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:50.505949 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf162e-6845-4e3e-bcd3-c79580f38695 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:50.506228 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf162e-6845-4e3e-bcd3-c79580f38695 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:50.506967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf162e-6845-4e3e-bcd3-c79580f38695 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:50.507276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf162e-6845-4e3e-bcd3-c79580f38695 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:50.508640 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf162e-6845-4e3e-bcd3-c79580f38695 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:50.520562 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 613/1266] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:50.526140 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb118ce1-659d-4af2-bf01-bdd758b5d0d9 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:50.621806 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb118ce1-659d-4af2-bf01-bdd758b5d0d9 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:50.640872 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:50.655312 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74b443-fddd-4480-a65a-7e94968dab95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:50.656102 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74b443-fddd-4480-a65a-7e94968dab95 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:50.656389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74b443-fddd-4480-a65a-7e94968dab95 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:50.656669 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74b443-fddd-4480-a65a-7e94968dab95 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:50.663950 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c74b443-fddd-4480-a65a-7e94968dab95 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:50.665389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c74b443-fddd-4480-a65a-7e94968dab95 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:50.679352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-4c74b443-fddd-4480-a65a-7e94968dab95 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 31 14:28:50.679352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:50.679352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:50.679352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:50.679352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:50.679352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:50.679352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:50.679352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:50.679352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:50.679352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:50.679352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:50.679352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:50.679352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:28:50.679352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:28:50.679352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:28:50.679352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:28:50.680453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:50.680453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:50.680453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:28:50.680453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:28:50.680453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:28:50.680453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:28:50.680453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:50.680453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:50.680453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:50.680453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:50.680453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:50.680453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:28:50.680453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:50.680453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:50.680453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:50.680453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:50.680453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:50.680453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:50.681488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:50.681488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:50.681488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:28:50.681488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:28:50.681488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:28:50.681488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:28:50.681488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:28:50.681488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:28:50.681488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 31 14:28:50.681488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:28:50.681488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 654/1267] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:28:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:50.686073 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf162e-6845-4e3e-bcd3-c79580f38695 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:50.782623 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf162e-6845-4e3e-bcd3-c79580f38695 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:50.816290 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7682c7-0eb5-4254-b311-d427290fec1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:50.816962 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7682c7-0eb5-4254-b311-d427290fec1b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:50.817243 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7682c7-0eb5-4254-b311-d427290fec1b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:50.817515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7682c7-0eb5-4254-b311-d427290fec1b None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:50.818345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7682c7-0eb5-4254-b311-d427290fec1b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:50.818676 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7682c7-0eb5-4254-b311-d427290fec1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:50.820099 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7682c7-0eb5-4254-b311-d427290fec1b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:50.828717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:50.837864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 614/1268] 10.222.0.67 () {56 vars in 1143 bytes} [Tue May 31 14:28:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:50.843626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74b443-fddd-4480-a65a-7e94968dab95 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:50.945508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74b443-fddd-4480-a65a-7e94968dab95 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:50.978987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc5247-ace0-4927-8ac5-bc4a16c7be6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:50.979768 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc5247-ace0-4927-8ac5-bc4a16c7be6d None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:50.980049 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc5247-ace0-4927-8ac5-bc4a16c7be6d None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:50.980321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc5247-ace0-4927-8ac5-bc4a16c7be6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:50.991006 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffc5247-ace0-4927-8ac5-bc4a16c7be6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:50.992439 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffc5247-ace0-4927-8ac5-bc4a16c7be6d None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:51.009442 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-5ffc5247-ace0-4927-8ac5-bc4a16c7be6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:51.009442 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:51.009442 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:51.009442 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:51.009442 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:51.009442 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:51.009442 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:51.009442 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:51.009442 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:51.009442 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:51.009442 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:51.009442 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:51.009442 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:28:51.009442 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:28:51.009442 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:28:51.010494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:28:51.010494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:51.010494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:51.010494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:51.010494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:51.010494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:51.010494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:51.010494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:51.010494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:28:51.010494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:51.010494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:51.010494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:51.010494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:51.010494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:51.010494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:51.010494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:51.010494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:51.011645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:28:51.011645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:28:51.011645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:51.011645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:28:51.011645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 655/1269] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:28:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:51.016407 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7682c7-0eb5-4254-b311-d427290fec1b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:51.111926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7682c7-0eb5-4254-b311-d427290fec1b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:51.145498 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc519e1-cee0-439d-8e10-6d68e5ae9b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:51.146160 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc519e1-cee0-439d-8e10-6d68e5ae9b79 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:51.146447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc519e1-cee0-439d-8e10-6d68e5ae9b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:51.146759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc519e1-cee0-439d-8e10-6d68e5ae9b79 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:51.147476 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc519e1-cee0-439d-8e10-6d68e5ae9b79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:51.147783 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc519e1-cee0-439d-8e10-6d68e5ae9b79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:51.149138 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc519e1-cee0-439d-8e10-6d68e5ae9b79 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:51.159986 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 615/1270] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:51.165510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc5247-ace0-4927-8ac5-bc4a16c7be6d None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:51.197315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:51.263159 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc5247-ace0-4927-8ac5-bc4a16c7be6d None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:51.295141 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672a81c-82a2-430c-83a1-ebf5974048c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:51.295915 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672a81c-82a2-430c-83a1-ebf5974048c9 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:51.296202 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672a81c-82a2-430c-83a1-ebf5974048c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:51.296480 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672a81c-82a2-430c-83a1-ebf5974048c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:51.303538 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8672a81c-82a2-430c-83a1-ebf5974048c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:51.304970 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8672a81c-82a2-430c-83a1-ebf5974048c9 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:51.318921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-8672a81c-82a2-430c-83a1-ebf5974048c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:28:51.318921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:51.318921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:51.318921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:51.318921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:51.318921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:51.318921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:51.318921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:51.318921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:51.318921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:51.318921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:51.318921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:51.318921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:28:51.318921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:28:51.318921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:28:51.319946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:28:51.319946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:51.319946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:51.319946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:51.319946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:51.319946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:51.319946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:51.319946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:51.319946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:28:51.319946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:51.319946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:51.319946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:51.319946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:51.319946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:51.319946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:51.319946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:51.319946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:51.321169 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:28:51.321169 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:28:51.321169 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:28:51.321169 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:28:51.321169 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:28:51.321169 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:28:51.321169 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:28:51.321169 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:28:51.321169 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 656/1271] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:28:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:51.325990 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc519e1-cee0-439d-8e10-6d68e5ae9b79 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:51.382278 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:51.423049 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc519e1-cee0-439d-8e10-6d68e5ae9b79 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:51.466375 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5fb00-14ad-48e9-aadf-b71cd0989fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:51.466990 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5fb00-14ad-48e9-aadf-b71cd0989fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:51.467275 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5fb00-14ad-48e9-aadf-b71cd0989fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:51.467552 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5fb00-14ad-48e9-aadf-b71cd0989fb2 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:51.468336 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5fb00-14ad-48e9-aadf-b71cd0989fb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:51.468645 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5fb00-14ad-48e9-aadf-b71cd0989fb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:51.470081 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5fb00-14ad-48e9-aadf-b71cd0989fb2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:51.485014 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 616/1272] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:28:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:51.491724 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672a81c-82a2-430c-83a1-ebf5974048c9 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:51.580271 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:51.589821 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672a81c-82a2-430c-83a1-ebf5974048c9 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:51.623527 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d14488-e7ed-47bf-afeb-1d54dca1f8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:51.624407 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d14488-e7ed-47bf-afeb-1d54dca1f8b9 None admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:51.624687 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d14488-e7ed-47bf-afeb-1d54dca1f8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:51.624964 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d14488-e7ed-47bf-afeb-1d54dca1f8b9 None admin] PATH_INFO: `/v3/projects/f965a93a2f674598890bd8cafededb2c/users/6c1c13515a8b41a781478dc65e6a88b5/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:51.626913 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d14488-e7ed-47bf-afeb-1d54dca1f8b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=f965a93a2f674598890bd8cafededb2c, user_id=6c1c13515a8b41a781478dc65e6a88b5, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:51.628384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d14488-e7ed-47bf-afeb-1d54dca1f8b9 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:51.640978 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 657/1273] 10.222.0.67 () {58 vars in 1439 bytes} [Tue May 31 14:28:51 2022] PUT /identity/v3/projects/f965a93a2f674598890bd8cafededb2c/users/6c1c13515a8b41a781478dc65e6a88b5/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:28:53.903553 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b5c18-2152-40ed-81b8-d94428d06196 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:53.904112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b5c18-2152-40ed-81b8-d94428d06196 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:53.904112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b5c18-2152-40ed-81b8-d94428d06196 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:53.905479 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 617/1274] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:28:53.910571 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8feeb9e-6790-4415-a0a8-81e055841315 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:53.910977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8feeb9e-6790-4415-a0a8-81e055841315 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:53.911157 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8feeb9e-6790-4415-a0a8-81e055841315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:53.928945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a8feeb9e-6790-4415-a0a8-81e055841315 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:28:53.930149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a8feeb9e-6790-4415-a0a8-81e055841315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:28:54.002874 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 658/1275] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:28:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 31 14:28:54.026605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b5c18-2152-40ed-81b8-d94428d06196 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:54.126885 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b5c18-2152-40ed-81b8-d94428d06196 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:54.145609 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:54.162100 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e34d0-0b52-49e6-8971-22d0bd9221ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:54.162857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e34d0-0b52-49e6-8971-22d0bd9221ef None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:54.163143 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e34d0-0b52-49e6-8971-22d0bd9221ef None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:54.163419 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e34d0-0b52-49e6-8971-22d0bd9221ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:54.173064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e34d0-0b52-49e6-8971-22d0bd9221ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 31 14:28:54.181092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e64e34d0-0b52-49e6-8971-22d0bd9221ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:28:54.181092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:54.181092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:54.181092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:54.181092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:54.181092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:54.181092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:54.181092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:54.181092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:54.181092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:54.181092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:54.181092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:54.181092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 31 14:28:54.181092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 31 14:28:54.181092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 31 14:28:54.181092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 31 14:28:54.182230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 31 14:28:54.182230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 31 14:28:54.182230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 31 14:28:54.182230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 31 14:28:54.182230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:54.182230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:54.182230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:54.182230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:54.182230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:54.182230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:54.182230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:54.182230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:28:54.182230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:54.182230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:54.182230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:54.182230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:54.183494 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:54.183494 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:54.183494 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:54.183494 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:54.183494 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 31 14:28:54.183494 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 31 14:28:54.183494 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 31 14:28:54.183494 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 31 14:28:54.183494 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 31 14:28:54.183494 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 31 14:28:54.183494 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 31 14:28:54.183494 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:28:54.183494 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 618/1276] 10.222.0.67 () {56 vars in 1096 bytes} [Tue May 31 14:28:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:54.188299 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8feeb9e-6790-4415-a0a8-81e055841315 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:54.291949 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:54.315670 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8feeb9e-6790-4415-a0a8-81e055841315 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:54.356098 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312682cc-73ac-4ed3-8125-d430686bfe70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:54.356803 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312682cc-73ac-4ed3-8125-d430686bfe70 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:54.357057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312682cc-73ac-4ed3-8125-d430686bfe70 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:54.357330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312682cc-73ac-4ed3-8125-d430686bfe70 None admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:54.358627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312682cc-73ac-4ed3-8125-d430686bfe70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:54.359152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312682cc-73ac-4ed3-8125-d430686bfe70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:54.360532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312682cc-73ac-4ed3-8125-d430686bfe70 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:54.371192 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 659/1277] 10.222.0.67 () {56 vars in 1099 bytes} [Tue May 31 14:28:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:54.378489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e34d0-0b52-49e6-8971-22d0bd9221ef None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:54.539599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e34d0-0b52-49e6-8971-22d0bd9221ef None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:54.587419 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1e3b9-5bf8-4981-a6f0-2d342a68258e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:54.588230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1e3b9-5bf8-4981-a6f0-2d342a68258e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:54.588526 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1e3b9-5bf8-4981-a6f0-2d342a68258e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:54.588801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1e3b9-5bf8-4981-a6f0-2d342a68258e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:54.600637 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1e3b9-5bf8-4981-a6f0-2d342a68258e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:54.602178 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1e3b9-5bf8-4981-a6f0-2d342a68258e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:54.620037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-bfb1e3b9-5bf8-4981-a6f0-2d342a68258e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:54.620037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:54.620037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:54.620037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:54.620037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:54.620037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:54.620037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:54.620037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:54.620037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:54.620037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:54.620037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:54.620037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:54.620037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:28:54.620037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:28:54.620037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:28:54.621335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:28:54.621335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:54.621335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:54.621335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:54.621335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:54.621335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:54.621335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:54.621335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:54.621335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:28:54.621335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:54.621335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:54.621335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:54.621335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:54.621335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:54.621335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:54.621335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:54.621335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:54.622805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:28:54.622805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:28:54.622805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:54.622805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:28:54.622805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 619/1278] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:28:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:54.627827 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312682cc-73ac-4ed3-8125-d430686bfe70 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:54.770930 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312682cc-73ac-4ed3-8125-d430686bfe70 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:54.820196 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bd521-2b51-4586-8c13-7d99139eba81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:54.820782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bd521-2b51-4586-8c13-7d99139eba81 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:54.821069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bd521-2b51-4586-8c13-7d99139eba81 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:54.821351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bd521-2b51-4586-8c13-7d99139eba81 None admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:54.822333 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15bd521-2b51-4586-8c13-7d99139eba81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:54.822662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15bd521-2b51-4586-8c13-7d99139eba81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:54.824573 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15bd521-2b51-4586-8c13-7d99139eba81 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:54.839060 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 660/1279] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:54.845367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1e3b9-5bf8-4981-a6f0-2d342a68258e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:54.886261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:54.975755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1e3b9-5bf8-4981-a6f0-2d342a68258e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:55.014144 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d22ca-6293-4bb5-bf4a-05ba6a88fe59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:55.014998 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d22ca-6293-4bb5-bf4a-05ba6a88fe59 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:55.015288 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d22ca-6293-4bb5-bf4a-05ba6a88fe59 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:55.015570 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d22ca-6293-4bb5-bf4a-05ba6a88fe59 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:55.024460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d22ca-6293-4bb5-bf4a-05ba6a88fe59 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:55.025922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d22ca-6293-4bb5-bf4a-05ba6a88fe59 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:55.048473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-0d9d22ca-6293-4bb5-bf4a-05ba6a88fe59 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 31 14:28:55.048473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:55.048473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:55.048473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:55.048473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:55.048473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:55.048473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:55.048473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:55.048473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:55.048473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:55.048473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:55.048473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:55.048473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 31 14:28:55.048473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 31 14:28:55.048473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 31 14:28:55.048473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 31 14:28:55.050298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:55.050298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:55.050298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 31 14:28:55.050298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:28:55.050298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 31 14:28:55.050298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 31 14:28:55.050298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:55.050298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:55.050298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:55.050298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:55.050298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:55.050298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:28:55.050298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:55.050298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:55.050298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:55.050298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:55.050298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:55.050298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:55.052425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:55.052425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:55.052425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 31 14:28:55.052425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 31 14:28:55.052425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 31 14:28:55.052425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 31 14:28:55.052425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 31 14:28:55.052425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 31 14:28:55.052425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 31 14:28:55.052425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:28:55.052425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 620/1280] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:28:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:55.056054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bd521-2b51-4586-8c13-7d99139eba81 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:55.060073 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:55.170107 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bd521-2b51-4586-8c13-7d99139eba81 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:55.216273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20cc25-9e5e-4994-b70f-cda70994e457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:55.217703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20cc25-9e5e-4994-b70f-cda70994e457 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:55.217996 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20cc25-9e5e-4994-b70f-cda70994e457 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:55.218279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20cc25-9e5e-4994-b70f-cda70994e457 None admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:55.219187 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee20cc25-9e5e-4994-b70f-cda70994e457 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:55.219506 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee20cc25-9e5e-4994-b70f-cda70994e457 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:55.221063 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee20cc25-9e5e-4994-b70f-cda70994e457 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:55.242427 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 661/1281] 10.222.0.67 () {56 vars in 1143 bytes} [Tue May 31 14:28:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:28:55.248579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d22ca-6293-4bb5-bf4a-05ba6a88fe59 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:55.348305 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:55.358204 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d22ca-6293-4bb5-bf4a-05ba6a88fe59 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:55.394190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371cbdbc-fc4b-4319-9a5e-c3d95cb3be03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:55.394820 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cbdbc-fc4b-4319-9a5e-c3d95cb3be03 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:55.395112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cbdbc-fc4b-4319-9a5e-c3d95cb3be03 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:55.395391 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cbdbc-fc4b-4319-9a5e-c3d95cb3be03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:55.464736 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371cbdbc-fc4b-4319-9a5e-c3d95cb3be03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:55.466220 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371cbdbc-fc4b-4319-9a5e-c3d95cb3be03 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:55.552322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:55.562883 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 621/1282] 10.222.0.67 () {58 vars in 1282 bytes} [Tue May 31 14:28:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 31 14:28:55.569286 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20cc25-9e5e-4994-b70f-cda70994e457 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:55.628889 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:55.666033 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20cc25-9e5e-4994-b70f-cda70994e457 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:55.699416 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da73bb1-0dc4-4a8f-86a5-974de3774f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:55.700196 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da73bb1-0dc4-4a8f-86a5-974de3774f5e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:55.700476 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da73bb1-0dc4-4a8f-86a5-974de3774f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:55.700747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da73bb1-0dc4-4a8f-86a5-974de3774f5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:55.710424 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da73bb1-0dc4-4a8f-86a5-974de3774f5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:55.711889 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da73bb1-0dc4-4a8f-86a5-974de3774f5e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:55.729990 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-6da73bb1-0dc4-4a8f-86a5-974de3774f5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:55.729990 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:55.729990 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:55.729990 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:55.729990 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:55.729990 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:55.729990 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:55.729990 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:55.729990 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:55.729990 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:55.729990 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:55.729990 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:55.729990 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 31 14:28:55.729990 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 31 14:28:55.729990 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 31 14:28:55.731069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 31 14:28:55.731069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:55.731069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:55.731069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:55.731069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:55.731069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:55.731069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:55.731069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:55.731069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:28:55.731069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:55.731069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:55.731069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:55.731069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:55.731069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:55.731069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:55.731069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:55.731069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:55.732043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 31 14:28:55.732043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 31 14:28:55.732043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 31 14:28:55.732043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:28:55.732043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 662/1283] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:28:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:55.736901 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371cbdbc-fc4b-4319-9a5e-c3d95cb3be03 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:55.834479 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371cbdbc-fc4b-4319-9a5e-c3d95cb3be03 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:55.868093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5f539-ef40-4cb5-a883-e8264a467008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:55.868757 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5f539-ef40-4cb5-a883-e8264a467008 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:55.869046 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5f539-ef40-4cb5-a883-e8264a467008 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:55.869328 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5f539-ef40-4cb5-a883-e8264a467008 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:55.870048 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5f539-ef40-4cb5-a883-e8264a467008 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:55.870360 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5f539-ef40-4cb5-a883-e8264a467008 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:55.871774 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5f539-ef40-4cb5-a883-e8264a467008 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:55.882823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 622/1284] 10.222.0.67 () {56 vars in 1109 bytes} [Tue May 31 14:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:55.888444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da73bb1-0dc4-4a8f-86a5-974de3774f5e None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:55.994037 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da73bb1-0dc4-4a8f-86a5-974de3774f5e None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:56.008382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:56.028506 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d95cf5-0c79-4bf4-99d5-070316276e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:56.029295 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d95cf5-0c79-4bf4-99d5-070316276e18 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:56.029577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d95cf5-0c79-4bf4-99d5-070316276e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:56.029852 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d95cf5-0c79-4bf4-99d5-070316276e18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:56.037722 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d95cf5-0c79-4bf4-99d5-070316276e18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:56.039488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d95cf5-0c79-4bf4-99d5-070316276e18 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:56.055248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application [None req-44d95cf5-0c79-4bf4-99d5-070316276e18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:28:56.055248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:56.055248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:56.055248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:56.055248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:56.055248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:56.055248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:56.055248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:56.055248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:56.055248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:56.055248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:56.055248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:56.055248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 31 14:28:56.055248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(project_id) May 31 14:28:56.055248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 31 14:28:56.056662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 31 14:28:56.056662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:56.056662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:56.056662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:56.056662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:56.056662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:56.056662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:56.056662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:56.056662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application with Lock( May 31 14:28:56.056662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:56.056662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:56.056662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:56.056662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:56.056662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:56.056662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:56.056662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:56.056662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:56.057800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 31 14:28:56.057800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 31 14:28:56.057800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 31 14:28:56.057800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 31 14:28:56.057800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 31 14:28:56.057800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 31 14:28:56.057800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 31 14:28:56.057800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: ERROR keystone.server.flask.application May 31 14:28:56.057800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 663/1285] 10.222.0.67 () {56 vars in 1111 bytes} [Tue May 31 14:28:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:56.062342 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5f539-ef40-4cb5-a883-e8264a467008 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:56.116490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:56.167034 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5f539-ef40-4cb5-a883-e8264a467008 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:56.200524 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4908eba1-3114-4bf2-a1b2-5861c339eba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:56.201186 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908eba1-3114-4bf2-a1b2-5861c339eba6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:56.201473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908eba1-3114-4bf2-a1b2-5861c339eba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:56.201755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908eba1-3114-4bf2-a1b2-5861c339eba6 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:56.202541 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4908eba1-3114-4bf2-a1b2-5861c339eba6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:56.202888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4908eba1-3114-4bf2-a1b2-5861c339eba6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:56.204313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4908eba1-3114-4bf2-a1b2-5861c339eba6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:56.216009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 623/1286] 10.222.0.67 () {56 vars in 1148 bytes} [Tue May 31 14:28:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:56.222080 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d95cf5-0c79-4bf4-99d5-070316276e18 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:56.334225 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d95cf5-0c79-4bf4-99d5-070316276e18 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:56.367761 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39cd4cc-2294-4bb5-9653-8dfb05e5869e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:56.368428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cd4cc-2294-4bb5-9653-8dfb05e5869e None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:56.368713 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cd4cc-2294-4bb5-9653-8dfb05e5869e None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:56.369019 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cd4cc-2294-4bb5-9653-8dfb05e5869e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:56.369865 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39cd4cc-2294-4bb5-9653-8dfb05e5869e None admin] RBAC: Adding query filter params (role.id=e8d679596ffb45939229d1e2a13ce9b0, scope.project.id=f965a93a2f674598890bd8cafededb2c, user.id=6c1c13515a8b41a781478dc65e6a88b5) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:56.370182 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39cd4cc-2294-4bb5-9653-8dfb05e5869e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:56.371646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39cd4cc-2294-4bb5-9653-8dfb05e5869e None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:56.380115 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 664/1287] 10.222.0.67 () {56 vars in 1374 bytes} [Tue May 31 14:28:56 2022] GET /identity/v3/role_assignments?user.id=6c1c13515a8b41a781478dc65e6a88b5&scope.project.id=f965a93a2f674598890bd8cafededb2c&role.id=e8d679596ffb45939229d1e2a13ce9b0 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:58.705777 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e3e80-de5f-409e-922f-95b40e5593f1 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:58.706337 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e3e80-de5f-409e-922f-95b40e5593f1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:58.706455 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e3e80-de5f-409e-922f-95b40e5593f1 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:58.707767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 624/1288] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:28:58.712915 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b89ff-57d3-4789-8e57-830936d8978e None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:58.713337 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b89ff-57d3-4789-8e57-830936d8978e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:58.713486 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b89ff-57d3-4789-8e57-830936d8978e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:58.730409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-b74b89ff-57d3-4789-8e57-830936d8978e None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:28:58.731651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b74b89ff-57d3-4789-8e57-830936d8978e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:28:58.760712 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:58.797035 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 665/1289] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:28:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:28:58.820085 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e3e80-de5f-409e-922f-95b40e5593f1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:58.930360 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e3e80-de5f-409e-922f-95b40e5593f1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:58.944854 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:28:58.973454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91105c47-cc05-4ead-8611-4bd57fc4669a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:58.974369 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91105c47-cc05-4ead-8611-4bd57fc4669a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:58.974779 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91105c47-cc05-4ead-8611-4bd57fc4669a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:58.975046 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91105c47-cc05-4ead-8611-4bd57fc4669a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:58.975782 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91105c47-cc05-4ead-8611-4bd57fc4669a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:58.977274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91105c47-cc05-4ead-8611-4bd57fc4669a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:58.991866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-91105c47-cc05-4ead-8611-4bd57fc4669a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 31 14:28:58.991866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:58.991866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:28:58.991866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:28:58.991866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:28:58.991866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:28:58.991866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:28:58.991866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:28:58.991866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 31 14:28:58.991866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:28:58.991866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:28:58.991866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:28:58.991866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:28:58.991866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:28:58.991866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:28:58.991866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:28:58.991866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:28:58.993110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:28:58.993110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:28:58.993110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:28:58.993110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:28:58.993110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:28:58.993110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:28:58.993110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:28:58.993110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:28:58.993110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:28:58.993110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:28:58.993110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:28:58.993110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:28:58.993110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:28:58.993110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:28:58.993110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:28:58.993110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:28:58.993991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:28:58.993991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:28:58.993991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:28:58.993991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:28:58.993991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:28:58.993991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:28:58.993991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:28:58.993991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:28:58.993991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:28:58.993991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:28:58.993991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:28:58.993991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:28:58.993991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 31 14:28:58.993991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:28:58.993991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 625/1290] 10.222.0.67 () {56 vars in 1117 bytes} [Tue May 31 14:28:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:28:59.000893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b89ff-57d3-4789-8e57-830936d8978e None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:59.108553 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b89ff-57d3-4789-8e57-830936d8978e None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:59.142747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce00b9cb-ebf5-45c0-ae14-aafc74690944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:59.143410 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00b9cb-ebf5-45c0-ae14-aafc74690944 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:59.144178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00b9cb-ebf5-45c0-ae14-aafc74690944 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:59.144450 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00b9cb-ebf5-45c0-ae14-aafc74690944 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:59.145186 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce00b9cb-ebf5-45c0-ae14-aafc74690944 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:59.145486 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce00b9cb-ebf5-45c0-ae14-aafc74690944 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:59.146888 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce00b9cb-ebf5-45c0-ae14-aafc74690944 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:59.151964 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 666/1291] 10.222.0.67 () {56 vars in 1116 bytes} [Tue May 31 14:28:58 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:28:59.157744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91105c47-cc05-4ead-8611-4bd57fc4669a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:28:59.258865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91105c47-cc05-4ead-8611-4bd57fc4669a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:28:59.295060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248db4c8-8415-427c-be9e-6f91198b00ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:28:59.295729 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248db4c8-8415-427c-be9e-6f91198b00ff None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:28:59.296012 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248db4c8-8415-427c-be9e-6f91198b00ff None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:28:59.296290 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248db4c8-8415-427c-be9e-6f91198b00ff None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:28:59.296996 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248db4c8-8415-427c-be9e-6f91198b00ff None admin] RBAC: Adding query filter params (type=placement) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:28:59.297365 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248db4c8-8415-427c-be9e-6f91198b00ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:28:59.298828 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248db4c8-8415-427c-be9e-6f91198b00ff None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:28:59.303849 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 626/1292] 10.222.0.67 () {56 vars in 1116 bytes} [Tue May 31 14:28:59 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:29:01.627028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406fe15-d528-4133-9f5a-ea8bf765b661 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:29:01.627570 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406fe15-d528-4133-9f5a-ea8bf765b661 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:29:01.627570 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406fe15-d528-4133-9f5a-ea8bf765b661 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:29:01.630457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 667/1293] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:29:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:29:01.636213 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7052a-5998-452b-93c3-cf5934bef83f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:29:01.636757 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7052a-5998-452b-93c3-cf5934bef83f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:29:01.636757 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7052a-5998-452b-93c3-cf5934bef83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:29:01.653577 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4cf7052a-5998-452b-93c3-cf5934bef83f None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:29:01.654839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4cf7052a-5998-452b-93c3-cf5934bef83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:29:01.684332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:29:01.723656 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 627/1294] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:29:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 31 14:29:01.747005 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406fe15-d528-4133-9f5a-ea8bf765b661 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:29:01.777160 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:29:01.847385 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406fe15-d528-4133-9f5a-ea8bf765b661 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:29:01.880504 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51f45d0-e381-4746-88b9-31f2a7b3e26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:29:01.881089 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f45d0-e381-4746-88b9-31f2a7b3e26b None admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:29:01.881371 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f45d0-e381-4746-88b9-31f2a7b3e26b None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:29:01.881646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f45d0-e381-4746-88b9-31f2a7b3e26b None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:29:01.882372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51f45d0-e381-4746-88b9-31f2a7b3e26b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:29:01.883783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51f45d0-e381-4746-88b9-31f2a7b3e26b None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:29:01.897429 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 668/1295] 10.222.0.67 () {60 vars in 1141 bytes} [Tue May 31 14:29:01 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:29:04.229163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194740cd-70ec-463a-aba2-131808dc4932 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:29:04.229660 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194740cd-70ec-463a-aba2-131808dc4932 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:29:04.229776 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194740cd-70ec-463a-aba2-131808dc4932 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:29:04.231185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 628/1296] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:29:04.236350 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da908b28-fa19-44bb-813a-fdca11138a21 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:29:04.236748 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da908b28-fa19-44bb-813a-fdca11138a21 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:29:04.236910 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da908b28-fa19-44bb-813a-fdca11138a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:29:04.253773 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-da908b28-fa19-44bb-813a-fdca11138a21 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:29:04.255011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-da908b28-fa19-44bb-813a-fdca11138a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:29:04.324979 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 669/1297] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:29:04 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:29:04.347980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194740cd-70ec-463a-aba2-131808dc4932 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:29:04.443851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194740cd-70ec-463a-aba2-131808dc4932 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:29:04.477442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc6bfd-d036-4254-b7fc-fc1a0028b968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:29:04.478253 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc6bfd-d036-4254-b7fc-fc1a0028b968 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:29:04.478538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc6bfd-d036-4254-b7fc-fc1a0028b968 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:29:04.478846 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc6bfd-d036-4254-b7fc-fc1a0028b968 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:29:04.479489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc6bfd-d036-4254-b7fc-fc1a0028b968 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:29:04.480846 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc6bfd-d036-4254-b7fc-fc1a0028b968 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:29:04.492325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-b4fc6bfd-d036-4254-b7fc-fc1a0028b968 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 31 14:29:04.492325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:29:04.492325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:29:04.492325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:29:04.492325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:29:04.492325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:29:04.492325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:29:04.492325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:29:04.492325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 31 14:29:04.492325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:29:04.492325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:29:04.492325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:29:04.492325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:29:04.492325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:29:04.492325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:29:04.492325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:29:04.492325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:29:04.493490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:29:04.493490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:29:04.493490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:29:04.493490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:29:04.493490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:29:04.493490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:29:04.493490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:29:04.493490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:29:04.493490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:29:04.493490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:29:04.493490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:29:04.493490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:29:04.493490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:29:04.493490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:29:04.493490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:29:04.493490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:29:04.494678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:29:04.494678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:29:04.494678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:29:04.494678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:29:04.494678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:29:04.494678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:29:04.494678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:29:04.494678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:29:04.494678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:29:04.494678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:29:04.494678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:29:04.494678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:29:04.494678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 31 14:29:04.494678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:29:04.494678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 629/1298] 10.222.0.67 () {56 vars in 1117 bytes} [Tue May 31 14:29:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:29:04.499369 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da908b28-fa19-44bb-813a-fdca11138a21 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:29:04.516610 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:29:04.593701 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da908b28-fa19-44bb-813a-fdca11138a21 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:29:04.625937 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd8bdb-b933-4dce-a1ed-5ae41f250d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:29:04.626591 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd8bdb-b933-4dce-a1ed-5ae41f250d00 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:29:04.626899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd8bdb-b933-4dce-a1ed-5ae41f250d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:29:04.627173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd8bdb-b933-4dce-a1ed-5ae41f250d00 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:29:04.627900 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd8bdb-b933-4dce-a1ed-5ae41f250d00 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:29:04.628208 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd8bdb-b933-4dce-a1ed-5ae41f250d00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:29:04.629559 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd8bdb-b933-4dce-a1ed-5ae41f250d00 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:29:04.634551 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 670/1299] 10.222.0.67 () {56 vars in 1116 bytes} [Tue May 31 14:29:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:29:04.640584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc6bfd-d036-4254-b7fc-fc1a0028b968 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:29:04.672424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:29:04.739896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc6bfd-d036-4254-b7fc-fc1a0028b968 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:29:04.772674 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2f282-56ac-428d-9f35-d4d774eb0e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:29:04.773326 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2f282-56ac-428d-9f35-d4d774eb0e78 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:29:04.773617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2f282-56ac-428d-9f35-d4d774eb0e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:29:04.773899 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2f282-56ac-428d-9f35-d4d774eb0e78 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:29:04.774711 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb2f282-56ac-428d-9f35-d4d774eb0e78 None admin] RBAC: Adding query filter params (interface=public, service_id=c3aab83a199140599e4547db9ebe4fa4, region_id=RegionOne) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:29:04.775035 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb2f282-56ac-428d-9f35-d4d774eb0e78 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:29:04.776396 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb2f282-56ac-428d-9f35-d4d774eb0e78 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:29:04.781334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 630/1300] 10.222.0.67 () {56 vars in 1251 bytes} [Tue May 31 14:29:04 2022] GET /identity/v3/endpoints?service_id=c3aab83a199140599e4547db9ebe4fa4&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:29:04.786830 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd8bdb-b933-4dce-a1ed-5ae41f250d00 None admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:29:04.864476 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:29:04.883066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd8bdb-b933-4dce-a1ed-5ae41f250d00 None admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:29:04.915613 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e97908-7814-4ed8-be02-6444e49d2324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:29:04.916262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e97908-7814-4ed8-be02-6444e49d2324 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:29:04.916552 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e97908-7814-4ed8-be02-6444e49d2324 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:29:04.916828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e97908-7814-4ed8-be02-6444e49d2324 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:29:04.917506 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e97908-7814-4ed8-be02-6444e49d2324 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:29:04.918896 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e97908-7814-4ed8-be02-6444e49d2324 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:29:04.925658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 671/1301] 10.222.0.67 () {56 vars in 1087 bytes} [Tue May 31 14:29:04 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:29:07.121806 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e59d8-c4a9-4c1e-8fba-55823280ccbd None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:29:07.122327 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e59d8-c4a9-4c1e-8fba-55823280ccbd None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:29:07.122446 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e59d8-c4a9-4c1e-8fba-55823280ccbd None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:29:07.123787 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 631/1302] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:29:07.128957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e4ff0-8ced-40ad-9253-bae834dcb9fa None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:29:07.129356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e4ff0-8ced-40ad-9253-bae834dcb9fa None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:29:07.129522 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e4ff0-8ced-40ad-9253-bae834dcb9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:29:07.145775 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-2c6e4ff0-8ced-40ad-9253-bae834dcb9fa None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:29:07.146995 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2c6e4ff0-8ced-40ad-9253-bae834dcb9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:29:07.211218 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 672/1303] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:29:07 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 31 14:29:07.233723 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e59d8-c4a9-4c1e-8fba-55823280ccbd None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:29:07.327205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:29:07.334796 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e59d8-c4a9-4c1e-8fba-55823280ccbd None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:29:07.367148 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33791d2-9741-4f98-acb6-a128573a25cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:29:07.367944 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33791d2-9741-4f98-acb6-a128573a25cb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:29:07.368230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33791d2-9741-4f98-acb6-a128573a25cb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:29:07.368508 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33791d2-9741-4f98-acb6-a128573a25cb None admin] PATH_INFO: `/v3/services/placement` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:29:07.369158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33791d2-9741-4f98-acb6-a128573a25cb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:29:07.370513 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33791d2-9741-4f98-acb6-a128573a25cb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:29:07.382224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-b33791d2-9741-4f98-acb6-a128573a25cb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 31 14:29:07.382224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:29:07.382224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 31 14:29:07.382224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 31 14:29:07.382224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 31 14:29:07.382224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 31 14:29:07.382224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 31 14:29:07.382224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:29:07.382224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 31 14:29:07.382224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:29:07.382224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 31 14:29:07.382224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:29:07.382224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 31 14:29:07.382224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 31 14:29:07.382224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 31 14:29:07.382224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 31 14:29:07.382224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 31 14:29:07.383402 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 31 14:29:07.383402 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 31 14:29:07.383402 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 31 14:29:07.383402 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 31 14:29:07.383402 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 31 14:29:07.383402 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 31 14:29:07.383402 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 31 14:29:07.383402 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 31 14:29:07.383402 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 31 14:29:07.383402 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 31 14:29:07.383402 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 31 14:29:07.383402 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 31 14:29:07.383402 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 31 14:29:07.383402 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 31 14:29:07.383402 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 31 14:29:07.383402 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 31 14:29:07.384450 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 31 14:29:07.384450 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 31 14:29:07.384450 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 31 14:29:07.384450 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 31 14:29:07.384450 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 31 14:29:07.384450 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 31 14:29:07.384450 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 31 14:29:07.384450 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 31 14:29:07.384450 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 31 14:29:07.384450 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 31 14:29:07.384450 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 31 14:29:07.384450 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 31 14:29:07.384450 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 31 14:29:07.384450 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 31 14:29:07.384450 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 632/1304] 10.222.0.67 () {56 vars in 1117 bytes} [Tue May 31 14:29:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 31 14:29:07.389532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e4ff0-8ced-40ad-9253-bae834dcb9fa None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:29:07.462254 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:29:07.485593 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e4ff0-8ced-40ad-9253-bae834dcb9fa None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:29:07.517758 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01861948-b630-4c5f-ba82-ccbc8a521950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:29:07.518929 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01861948-b630-4c5f-ba82-ccbc8a521950 None admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:29:07.519217 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01861948-b630-4c5f-ba82-ccbc8a521950 None admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:29:07.519495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01861948-b630-4c5f-ba82-ccbc8a521950 None admin] PATH_INFO: `/v3/services` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:29:07.520203 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01861948-b630-4c5f-ba82-ccbc8a521950 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:29:07.520513 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01861948-b630-4c5f-ba82-ccbc8a521950 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:29:07.521884 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01861948-b630-4c5f-ba82-ccbc8a521950 None admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:29:07.526933 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 673/1305] 10.222.0.67 () {56 vars in 1116 bytes} [Tue May 31 14:29:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:29:07.532515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33791d2-9741-4f98-acb6-a128573a25cb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:29:07.629454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33791d2-9741-4f98-acb6-a128573a25cb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:29:07.662725 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7178b-fbf1-4b00-a795-8ba4e09a22fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:29:07.663325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7178b-fbf1-4b00-a795-8ba4e09a22fd None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:29:07.663609 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7178b-fbf1-4b00-a795-8ba4e09a22fd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:29:07.663887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7178b-fbf1-4b00-a795-8ba4e09a22fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:29:07.664626 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7178b-fbf1-4b00-a795-8ba4e09a22fd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:29:07.665980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7178b-fbf1-4b00-a795-8ba4e09a22fd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:29:07.684139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 633/1306] 10.222.0.67 () {60 vars in 1144 bytes} [Tue May 31 14:29:07 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:29:17.524202 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=30 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:29:17.680091 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:30:11.332598 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bacef4-dc94-4cab-b812-e0afae52064d None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:11.333476 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bacef4-dc94-4cab-b812-e0afae52064d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:11.333921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bacef4-dc94-4cab-b812-e0afae52064d None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:11.335478 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 674/1307] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:30:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:30:11.340841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb39a85-79e4-4bde-b2cb-94abd40cc0d4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:11.341261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb39a85-79e4-4bde-b2cb-94abd40cc0d4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:11.341413 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb39a85-79e4-4bde-b2cb-94abd40cc0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:11.436555 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2fb39a85-79e4-4bde-b2cb-94abd40cc0d4 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:30:11.438615 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2fb39a85-79e4-4bde-b2cb-94abd40cc0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:11.738776 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 634/1308] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:30:11.792670 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6d196-e693-4752-b20c-3aebc54463fb None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:11.793245 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6d196-e693-4752-b20c-3aebc54463fb None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:11.793670 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6d196-e693-4752-b20c-3aebc54463fb None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:11.795231 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 675/1309] 10.222.0.67 () {54 vars in 964 bytes} [Tue May 31 14:30:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:30:11.810311 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639bf79-567d-42ba-a964-57f00712c9c9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:11.810907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639bf79-567d-42ba-a964-57f00712c9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:11.811412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639bf79-567d-42ba-a964-57f00712c9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:11.971848 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3639bf79-567d-42ba-a964-57f00712c9c9 None None] MFA Rules not processed for user `8966bc2ea7ff497c84214c9c9e6d53a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:30:11.973122 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3639bf79-567d-42ba-a964-57f00712c9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:12.020284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:30:12.157224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 635/1310] 10.222.0.67 () {58 vars in 1061 bytes} [Tue May 31 14:30:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:30:12.166572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6d196-e693-4752-b20c-3aebc54463fb None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:30:12.168645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-42a6d196-e693-4752-b20c-3aebc54463fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:12.362989 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6d196-e693-4752-b20c-3aebc54463fb None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:30:12.457855 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54fa7b-73e5-4852-a52b-09181889c662 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:30:12.458533 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54fa7b-73e5-4852-a52b-09181889c662 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:12.458861 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54fa7b-73e5-4852-a52b-09181889c662 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:12.459144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54fa7b-73e5-4852-a52b-09181889c662 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:12.460860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3a54fa7b-73e5-4852-a52b-09181889c662 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:12.517749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:30:12.577964 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54fa7b-73e5-4852-a52b-09181889c662 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:30:12.580012 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54fa7b-73e5-4852-a52b-09181889c662 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:30:12.600680 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 676/1311] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:30:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:30:15.283225 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916bed1-62d9-421b-9897-a372464d047f None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:15.283724 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916bed1-62d9-421b-9897-a372464d047f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:15.283724 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916bed1-62d9-421b-9897-a372464d047f None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:15.285249 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 636/1312] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:30:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:30:15.290318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d107da86-8519-465c-b345-64748bb7846a None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:15.290774 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d107da86-8519-465c-b345-64748bb7846a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:15.290930 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d107da86-8519-465c-b345-64748bb7846a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:15.361539 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-d107da86-8519-465c-b345-64748bb7846a None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:30:15.362768 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d107da86-8519-465c-b345-64748bb7846a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:15.459408 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 677/1313] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:30:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:30:15.497860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916bed1-62d9-421b-9897-a372464d047f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:30:15.499439 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a916bed1-62d9-421b-9897-a372464d047f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:15.601873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916bed1-62d9-421b-9897-a372464d047f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:30:15.674256 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:30:15.677133 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e5056-5b93-4315-8f1b-ce59e37904ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:30:15.677799 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e5056-5b93-4315-8f1b-ce59e37904ec service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:15.678086 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e5056-5b93-4315-8f1b-ce59e37904ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:15.678366 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e5056-5b93-4315-8f1b-ce59e37904ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:15.680571 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-999e5056-5b93-4315-8f1b-ce59e37904ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:15.821717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e5056-5b93-4315-8f1b-ce59e37904ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:30:15.823404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e5056-5b93-4315-8f1b-ce59e37904ec service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:30:15.847023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 637/1314] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:30:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:30:18.468231 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbc39f-4490-48aa-b9e3-e21151e05275 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:18.469195 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbc39f-4490-48aa-b9e3-e21151e05275 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:18.469338 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbc39f-4490-48aa-b9e3-e21151e05275 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:18.471461 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 678/1315] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:30:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:30:18.477243 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b007566-6f26-44c0-8524-9d72f53211ef None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:18.477744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b007566-6f26-44c0-8524-9d72f53211ef None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:18.477839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b007566-6f26-44c0-8524-9d72f53211ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:18.548448 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7b007566-6f26-44c0-8524-9d72f53211ef None None] MFA Rules not processed for user `f89c7583bd684f1a883ca3ccb2230e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:30:18.549679 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7b007566-6f26-44c0-8524-9d72f53211ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:18.601035 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:30:18.644766 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 638/1316] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:30:18 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:30:18.685735 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecbc39f-4490-48aa-b9e3-e21151e05275 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:30:18.687333 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-aecbc39f-4490-48aa-b9e3-e21151e05275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:18.697393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:30:18.790609 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecbc39f-4490-48aa-b9e3-e21151e05275 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:30:18.861899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f016bbf-4b72-4f1b-896c-3b4c39806240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:30:18.862549 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f016bbf-4b72-4f1b-896c-3b4c39806240 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:18.862871 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f016bbf-4b72-4f1b-896c-3b4c39806240 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:18.863163 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f016bbf-4b72-4f1b-896c-3b4c39806240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:18.864821 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9f016bbf-4b72-4f1b-896c-3b4c39806240 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:18.926809 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:30:18.973875 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f016bbf-4b72-4f1b-896c-3b4c39806240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:30:18.975356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f016bbf-4b72-4f1b-896c-3b4c39806240 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:30:18.993428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 679/1317] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:30:18 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:30:21.775611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509710f-66a5-401b-b343-d42a1f82f19c None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:21.776206 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509710f-66a5-401b-b343-d42a1f82f19c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:21.776206 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509710f-66a5-401b-b343-d42a1f82f19c None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:21.777541 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 639/1318] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:30:21.782935 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12909ab-68ec-442d-85b2-dc9b916f3a3f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:21.783360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12909ab-68ec-442d-85b2-dc9b916f3a3f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:21.783507 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12909ab-68ec-442d-85b2-dc9b916f3a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:21.848873 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a12909ab-68ec-442d-85b2-dc9b916f3a3f None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:30:21.850085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a12909ab-68ec-442d-85b2-dc9b916f3a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:21.948647 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 680/1319] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:30:21 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:30:21.987479 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509710f-66a5-401b-b343-d42a1f82f19c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:30:21.989001 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0509710f-66a5-401b-b343-d42a1f82f19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:22.090874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509710f-66a5-401b-b343-d42a1f82f19c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:30:22.162951 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb088d6-3077-4d2a-b4d9-dc6d1c8e6d81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:30:22.163603 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb088d6-3077-4d2a-b4d9-dc6d1c8e6d81 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:22.163884 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb088d6-3077-4d2a-b4d9-dc6d1c8e6d81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:22.164158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb088d6-3077-4d2a-b4d9-dc6d1c8e6d81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:22.165801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-afb088d6-3077-4d2a-b4d9-dc6d1c8e6d81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:22.173731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:30:22.273064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb088d6-3077-4d2a-b4d9-dc6d1c8e6d81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:30:22.274498 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb088d6-3077-4d2a-b4d9-dc6d1c8e6d81 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:30:22.292556 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 640/1320] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:30:21 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:30:25.216510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7b661-fdb2-4e2c-8575-f10a4df3c313 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:25.217094 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7b661-fdb2-4e2c-8575-f10a4df3c313 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:25.217094 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7b661-fdb2-4e2c-8575-f10a4df3c313 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:25.219346 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 681/1321] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:30:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:30:25.224678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b7a79-8e0f-4be6-bff4-a3f44027327a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:25.225074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b7a79-8e0f-4be6-bff4-a3f44027327a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:25.225249 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b7a79-8e0f-4be6-bff4-a3f44027327a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:25.291766 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-bb0b7a79-8e0f-4be6-bff4-a3f44027327a None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:30:25.292972 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bb0b7a79-8e0f-4be6-bff4-a3f44027327a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:25.352351 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:30:25.393540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 641/1322] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:30:25.428685 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7b661-fdb2-4e2c-8575-f10a4df3c313 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:30:25.430226 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-30a7b661-fdb2-4e2c-8575-f10a4df3c313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:25.452382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:30:25.536301 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7b661-fdb2-4e2c-8575-f10a4df3c313 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:30:25.609735 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ecc24-2bdc-499f-9473-1b7c51dde2e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:30:25.610379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ecc24-2bdc-499f-9473-1b7c51dde2e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:25.610676 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ecc24-2bdc-499f-9473-1b7c51dde2e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:25.610962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ecc24-2bdc-499f-9473-1b7c51dde2e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:25.612600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-696ecc24-2bdc-499f-9473-1b7c51dde2e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:25.688547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:30:25.727253 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ecc24-2bdc-499f-9473-1b7c51dde2e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:30:25.728729 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ecc24-2bdc-499f-9473-1b7c51dde2e6 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:30:25.747537 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 682/1323] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:30:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:30:28.059208 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39bef2-a4a6-4a5e-a5a6-d044c10fafe4 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:28.059787 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39bef2-a4a6-4a5e-a5a6-d044c10fafe4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:28.059787 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39bef2-a4a6-4a5e-a5a6-d044c10fafe4 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:28.061294 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 642/1324] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:30:28.066437 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2ebdb-bdf7-43ac-bc52-3c13583f8331 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:28.066897 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2ebdb-bdf7-43ac-bc52-3c13583f8331 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:28.067038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2ebdb-bdf7-43ac-bc52-3c13583f8331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:28.135860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-4ee2ebdb-bdf7-43ac-bc52-3c13583f8331 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:30:28.137103 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4ee2ebdb-bdf7-43ac-bc52-3c13583f8331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:28.237578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 683/1325] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:30:28.273243 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39bef2-a4a6-4a5e-a5a6-d044c10fafe4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:30:28.274883 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1a39bef2-a4a6-4a5e-a5a6-d044c10fafe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:28.378922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39bef2-a4a6-4a5e-a5a6-d044c10fafe4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:30:28.451096 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b340cdfa-14eb-4a81-ab51-edc1cb3f80cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:30:28.451740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340cdfa-14eb-4a81-ab51-edc1cb3f80cd service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:28.452018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340cdfa-14eb-4a81-ab51-edc1cb3f80cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:28.452290 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340cdfa-14eb-4a81-ab51-edc1cb3f80cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:28.453907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b340cdfa-14eb-4a81-ab51-edc1cb3f80cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:28.470749 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:30:28.563731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b340cdfa-14eb-4a81-ab51-edc1cb3f80cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:30:28.565156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b340cdfa-14eb-4a81-ab51-edc1cb3f80cd service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:30:28.583219 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 643/1326] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:30:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:30:30.917587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e643dca-26a4-4273-84bf-bcc33101df21 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:30.918131 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e643dca-26a4-4273-84bf-bcc33101df21 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:30.918260 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e643dca-26a4-4273-84bf-bcc33101df21 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:30.919713 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 684/1327] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:30:30.925364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc924a-8723-4669-935c-4de2f074d3c4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:30.925781 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc924a-8723-4669-935c-4de2f074d3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:30.925986 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc924a-8723-4669-935c-4de2f074d3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:31.002419 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2dbc924a-8723-4669-935c-4de2f074d3c4 None None] MFA Rules not processed for user `f89c7583bd684f1a883ca3ccb2230e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:30:31.003818 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2dbc924a-8723-4669-935c-4de2f074d3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:31.077687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:30:31.114696 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 644/1328] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:30:30 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:30:31.151988 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e643dca-26a4-4273-84bf-bcc33101df21 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:30:31.153591 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3e643dca-26a4-4273-84bf-bcc33101df21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:31.188995 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:30:31.270281 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e643dca-26a4-4273-84bf-bcc33101df21 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:30:31.357431 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ac98c-f0b0-4393-bee9-1450b5de7669 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:30:31.358095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ac98c-f0b0-4393-bee9-1450b5de7669 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:31.358385 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ac98c-f0b0-4393-bee9-1450b5de7669 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:31.358674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ac98c-f0b0-4393-bee9-1450b5de7669 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:31.361205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8f6ac98c-f0b0-4393-bee9-1450b5de7669 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:31.448908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:30:31.487115 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6ac98c-f0b0-4393-bee9-1450b5de7669 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:30:31.488581 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6ac98c-f0b0-4393-bee9-1450b5de7669 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:30:31.508088 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 685/1329] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:30:31 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:30:34.216058 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92948187-2c55-49a9-ab59-0f6a001187f1 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:34.216579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92948187-2c55-49a9-ab59-0f6a001187f1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:34.216677 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92948187-2c55-49a9-ab59-0f6a001187f1 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:34.218000 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 645/1330] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:30:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:30:34.223547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac0f0e-09d1-4348-8a57-739995bf6418 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:34.223960 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac0f0e-09d1-4348-8a57-739995bf6418 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:34.224075 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac0f0e-09d1-4348-8a57-739995bf6418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:34.430106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-f2ac0f0e-09d1-4348-8a57-739995bf6418 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:30:34.431390 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f2ac0f0e-09d1-4348-8a57-739995bf6418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:34.549134 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 686/1331] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:30:34.584274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92948187-2c55-49a9-ab59-0f6a001187f1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:30:34.585820 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-92948187-2c55-49a9-ab59-0f6a001187f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:34.712019 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92948187-2c55-49a9-ab59-0f6a001187f1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:30:34.803522 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a67388-8476-44ae-9bdf-f9569d667200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:30:34.804180 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a67388-8476-44ae-9bdf-f9569d667200 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:34.804774 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a67388-8476-44ae-9bdf-f9569d667200 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:34.805104 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a67388-8476-44ae-9bdf-f9569d667200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:34.807742 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-60a67388-8476-44ae-9bdf-f9569d667200 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:34.853739 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:30:34.965231 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a67388-8476-44ae-9bdf-f9569d667200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:30:34.966766 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a67388-8476-44ae-9bdf-f9569d667200 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:30:34.988726 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 646/1332] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:30:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:30:37.307858 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658cc6c-aac5-4f2e-9d56-69c0020a2af2 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:37.308341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658cc6c-aac5-4f2e-9d56-69c0020a2af2 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:37.308341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658cc6c-aac5-4f2e-9d56-69c0020a2af2 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:37.309729 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 687/1333] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:30:37.315665 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ad0a4-bebe-431d-9abb-6b3397b3cdc0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:37.316093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ad0a4-bebe-431d-9abb-6b3397b3cdc0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:37.316243 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ad0a4-bebe-431d-9abb-6b3397b3cdc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:37.381033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-779ad0a4-bebe-431d-9abb-6b3397b3cdc0 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:30:37.382179 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-779ad0a4-bebe-431d-9abb-6b3397b3cdc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:37.456486 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:30:37.479946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 647/1334] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:30:37.518985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658cc6c-aac5-4f2e-9d56-69c0020a2af2 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:30:37.520497 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1658cc6c-aac5-4f2e-9d56-69c0020a2af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:37.561751 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:30:37.625575 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658cc6c-aac5-4f2e-9d56-69c0020a2af2 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:30:37.697758 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24cd67-3447-49d7-90e1-6d16f0eca523 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:30:37.698322 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24cd67-3447-49d7-90e1-6d16f0eca523 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:37.698604 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24cd67-3447-49d7-90e1-6d16f0eca523 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:37.698906 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24cd67-3447-49d7-90e1-6d16f0eca523 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:37.700530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-fa24cd67-3447-49d7-90e1-6d16f0eca523 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:37.779843 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:30:37.806939 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24cd67-3447-49d7-90e1-6d16f0eca523 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:30:37.808375 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24cd67-3447-49d7-90e1-6d16f0eca523 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:30:37.826705 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 688/1335] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:30:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:30:40.397405 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84cae70-7171-458a-a1cd-f6169111b6ed None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:40.397931 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84cae70-7171-458a-a1cd-f6169111b6ed None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:40.397931 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84cae70-7171-458a-a1cd-f6169111b6ed None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:40.399308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 648/1336] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:30:40.404512 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085836f-fcef-4338-b482-7cb3f9b89e29 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:40.404895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085836f-fcef-4338-b482-7cb3f9b89e29 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:40.405080 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085836f-fcef-4338-b482-7cb3f9b89e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:40.470052 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6085836f-fcef-4338-b482-7cb3f9b89e29 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:30:40.471275 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6085836f-fcef-4338-b482-7cb3f9b89e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:40.569428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 689/1337] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:30:40 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:30:40.620852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79559e8f-aa6b-46ca-96b8-d09dd1c35f35 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:40.621136 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79559e8f-aa6b-46ca-96b8-d09dd1c35f35 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:40.621440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79559e8f-aa6b-46ca-96b8-d09dd1c35f35 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:40.622866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 649/1338] 10.222.0.67 () {54 vars in 964 bytes} [Tue May 31 14:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:30:40.628799 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b6d4b-31a3-4afc-abe6-cff7de074ef1 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:40.629086 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b6d4b-31a3-4afc-abe6-cff7de074ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:40.629359 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b6d4b-31a3-4afc-abe6-cff7de074ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:40.684866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:30:40.708095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-dc2b6d4b-31a3-4afc-abe6-cff7de074ef1 None None] MFA Rules not processed for user `8966bc2ea7ff497c84214c9c9e6d53a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:30:40.709319 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-dc2b6d4b-31a3-4afc-abe6-cff7de074ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:40.784558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 690/1339] 10.222.0.67 () {58 vars in 1061 bytes} [Tue May 31 14:30:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:30:40.793679 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79559e8f-aa6b-46ca-96b8-d09dd1c35f35 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:30:40.795231 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-79559e8f-aa6b-46ca-96b8-d09dd1c35f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:40.897691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79559e8f-aa6b-46ca-96b8-d09dd1c35f35 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:30:40.986404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb0c40-2ada-47d4-86f8-ff86be625ee3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:30:40.987016 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb0c40-2ada-47d4-86f8-ff86be625ee3 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:40.987302 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb0c40-2ada-47d4-86f8-ff86be625ee3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:40.987580 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb0c40-2ada-47d4-86f8-ff86be625ee3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:40.989466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-09eb0c40-2ada-47d4-86f8-ff86be625ee3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:41.024704 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:30:41.097244 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb0c40-2ada-47d4-86f8-ff86be625ee3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:30:41.098704 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb0c40-2ada-47d4-86f8-ff86be625ee3 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:30:41.116034 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 650/1340] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:30:40 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:30:45.857056 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a2acf-da82-4c33-8cd4-7b867f3dd225 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:45.857583 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a2acf-da82-4c33-8cd4-7b867f3dd225 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:45.857583 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a2acf-da82-4c33-8cd4-7b867f3dd225 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:45.859052 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 691/1341] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:30:45.864979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0b8c7-ac3e-45a3-bed9-17cf7fb75ddb None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:45.865422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0b8c7-ac3e-45a3-bed9-17cf7fb75ddb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:45.865562 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0b8c7-ac3e-45a3-bed9-17cf7fb75ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:45.939995 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-50e0b8c7-ac3e-45a3-bed9-17cf7fb75ddb None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:30:45.941133 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-50e0b8c7-ac3e-45a3-bed9-17cf7fb75ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:46.033510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:30:46.047734 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 651/1342] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:30:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:30:46.086528 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a2acf-da82-4c33-8cd4-7b867f3dd225 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:30:46.088861 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9a7a2acf-da82-4c33-8cd4-7b867f3dd225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:46.186344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a2acf-da82-4c33-8cd4-7b867f3dd225 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:30:46.189965 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697bc668-9eaa-4ba6-8203-10aa1ceed718 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:30:46.190676 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bc668-9eaa-4ba6-8203-10aa1ceed718 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:46.190929 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bc668-9eaa-4ba6-8203-10aa1ceed718 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:46.191203 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bc668-9eaa-4ba6-8203-10aa1ceed718 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:46.200381 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697bc668-9eaa-4ba6-8203-10aa1ceed718 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:30:46.201806 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697bc668-9eaa-4ba6-8203-10aa1ceed718 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:30:46.211712 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 692/1343] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:30:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:30:49.127538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0620779-33a3-4a5c-bd44-003940cd1fe7 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:49.128049 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0620779-33a3-4a5c-bd44-003940cd1fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:49.128151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0620779-33a3-4a5c-bd44-003940cd1fe7 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:49.129750 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 652/1344] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:30:49.135018 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b2a12-3bdf-48bd-a02d-32c8c4f702da None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:49.135408 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b2a12-3bdf-48bd-a02d-32c8c4f702da None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:49.135626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b2a12-3bdf-48bd-a02d-32c8c4f702da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:49.164126 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-ca4b2a12-3bdf-48bd-a02d-32c8c4f702da None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:30:49.165319 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ca4b2a12-3bdf-48bd-a02d-32c8c4f702da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:49.196899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:30:49.250546 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 693/1345] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:30:49 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:30:49.288247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0620779-33a3-4a5c-bd44-003940cd1fe7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:30:49.298853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0620779-33a3-4a5c-bd44-003940cd1fe7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:30:49.302388 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b17ac-8b00-4097-b65e-b266a026e5f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:30:49.303065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b17ac-8b00-4097-b65e-b266a026e5f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:49.303352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b17ac-8b00-4097-b65e-b266a026e5f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:49.303630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b17ac-8b00-4097-b65e-b266a026e5f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:49.312561 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b17ac-8b00-4097-b65e-b266a026e5f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:30:49.313984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b17ac-8b00-4097-b65e-b266a026e5f1 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:30:49.323632 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 653/1346] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:30:49 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:30:53.485236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee637c-a56d-4c66-a44e-2040c5427651 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:53.485749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee637c-a56d-4c66-a44e-2040c5427651 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:53.485853 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee637c-a56d-4c66-a44e-2040c5427651 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:53.487214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 694/1347] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:30:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:30:53.492480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f624a-89f8-4f1b-b66b-ecf3f4ac49e1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:53.492917 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f624a-89f8-4f1b-b66b-ecf3f4ac49e1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:53.493029 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f624a-89f8-4f1b-b66b-ecf3f4ac49e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:53.511275 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c49f624a-89f8-4f1b-b66b-ecf3f4ac49e1 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:30:53.512473 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c49f624a-89f8-4f1b-b66b-ecf3f4ac49e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:53.522181 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 654/1348] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:30:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:30:53.558087 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee637c-a56d-4c66-a44e-2040c5427651 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:30:53.569480 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee637c-a56d-4c66-a44e-2040c5427651 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:30:53.573304 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226e19d-8ccd-4c34-b64b-aa915289a57c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:30:53.573893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226e19d-8ccd-4c34-b64b-aa915289a57c service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:53.574179 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226e19d-8ccd-4c34-b64b-aa915289a57c service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:53.574460 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226e19d-8ccd-4c34-b64b-aa915289a57c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:53.583510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c226e19d-8ccd-4c34-b64b-aa915289a57c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:30:53.584947 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c226e19d-8ccd-4c34-b64b-aa915289a57c service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:30:53.594360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 695/1349] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:30:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:30:59.029216 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337c055-c4fa-46fb-a1cb-2dac9d56ac22 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:59.030051 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337c055-c4fa-46fb-a1cb-2dac9d56ac22 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:59.030503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337c055-c4fa-46fb-a1cb-2dac9d56ac22 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:59.033172 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 655/1350] 10.222.0.67 () {54 vars in 921 bytes} [Tue May 31 14:30:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:30:59.038342 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d5609-f622-48d0-b102-7026b21a9ce4 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:30:59.038779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d5609-f622-48d0-b102-7026b21a9ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:30:59.038948 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d5609-f622-48d0-b102-7026b21a9ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:30:59.119856 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-c03d5609-f622-48d0-b102-7026b21a9ce4 None None] MFA Rules not processed for user `6c1c13515a8b41a781478dc65e6a88b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:30:59.119856 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c03d5609-f622-48d0-b102-7026b21a9ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:30:59.200785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 696/1351] 10.222.0.67 () {58 vars in 1018 bytes} [Tue May 31 14:30:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:31:03.506181 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=13 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:31:03.558416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84c9a9-caf0-4b24-8438-bdd2f110c9e6 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:03.558895 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84c9a9-caf0-4b24-8438-bdd2f110c9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:03.559120 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84c9a9-caf0-4b24-8438-bdd2f110c9e6 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:03.560826 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 656/1352] 10.222.0.67 () {54 vars in 921 bytes} [Tue May 31 14:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:31:03.567340 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f5f27-66b7-4406-bfe4-aeef4a33953e None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:03.567907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f5f27-66b7-4406-bfe4-aeef4a33953e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:03.567907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f5f27-66b7-4406-bfe4-aeef4a33953e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:03.587661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-2d5f5f27-66b7-4406-bfe4-aeef4a33953e None None] MFA Rules not processed for user `6c1c13515a8b41a781478dc65e6a88b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:31:03.589019 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2d5f5f27-66b7-4406-bfe4-aeef4a33953e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:31:03.598424 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 697/1353] 10.222.0.67 () {58 vars in 1018 bytes} [Tue May 31 14:31:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:31:05.152567 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6e61c-2193-4016-8d73-1dff80b49b72 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:05.153072 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6e61c-2193-4016-8d73-1dff80b49b72 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:05.153182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6e61c-2193-4016-8d73-1dff80b49b72 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:05.154670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 657/1354] 10.222.0.67 () {54 vars in 921 bytes} [Tue May 31 14:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:31:05.160415 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6a5d8-72c1-43cd-b0d6-7d444de1e96e None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:05.160878 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6a5d8-72c1-43cd-b0d6-7d444de1e96e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:05.161003 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6a5d8-72c1-43cd-b0d6-7d444de1e96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:05.180441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6ec6a5d8-72c1-43cd-b0d6-7d444de1e96e None None] MFA Rules not processed for user `6c1c13515a8b41a781478dc65e6a88b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:31:05.182475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6ec6a5d8-72c1-43cd-b0d6-7d444de1e96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:31:05.196785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 698/1355] 10.222.0.67 () {58 vars in 1018 bytes} [Tue May 31 14:31:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:31:09.089743 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18366fe5-ea35-4898-9121-8de30b58d0db None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:09.090261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18366fe5-ea35-4898-9121-8de30b58d0db None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:09.090463 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18366fe5-ea35-4898-9121-8de30b58d0db None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:09.091965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 658/1356] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:31:09.097065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a57e2-4f79-4878-a715-e04a1c1f6401 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:09.097646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a57e2-4f79-4878-a715-e04a1c1f6401 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:09.097927 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a57e2-4f79-4878-a715-e04a1c1f6401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:09.115616 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-686a57e2-4f79-4878-a715-e04a1c1f6401 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:31:09.116837 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-686a57e2-4f79-4878-a715-e04a1c1f6401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:31:09.131499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 699/1357] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:31:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:31:09.191245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daefad3-569a-413b-a464-848e8a902941 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:09.192098 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daefad3-569a-413b-a464-848e8a902941 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:09.192626 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daefad3-569a-413b-a464-848e8a902941 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:09.194235 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 659/1358] 10.222.0.67 () {54 vars in 961 bytes} [Tue May 31 14:31:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:31:09.204954 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd23879-2776-4657-8ca5-129df513863d None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:09.206581 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd23879-2776-4657-8ca5-129df513863d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:09.207333 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd23879-2776-4657-8ca5-129df513863d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:09.256350 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:31:09.265002 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-0bd23879-2776-4657-8ca5-129df513863d None None] MFA Rules not processed for user `68c6363b60954c3ebf50d2019f5f2fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:31:09.266355 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0bd23879-2776-4657-8ca5-129df513863d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:31:09.352625 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 700/1359] 10.222.0.67 () {58 vars in 1058 bytes} [Tue May 31 14:31:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:31:09.361016 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daefad3-569a-413b-a464-848e8a902941 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:31:09.376257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daefad3-569a-413b-a464-848e8a902941 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:31:09.381901 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce3692-6c29-4ade-9487-ea166ee4afd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:31:09.382704 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce3692-6c29-4ade-9487-ea166ee4afd6 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:09.383096 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce3692-6c29-4ade-9487-ea166ee4afd6 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:09.383581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce3692-6c29-4ade-9487-ea166ee4afd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:09.392237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce3692-6c29-4ade-9487-ea166ee4afd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:31:09.393943 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce3692-6c29-4ade-9487-ea166ee4afd6 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:31:09.404320 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 660/1360] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:31:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:31:11.883863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af21d8-6a9d-4d16-a08c-7414e6d2c5f8 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:11.884424 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af21d8-6a9d-4d16-a08c-7414e6d2c5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:11.884424 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af21d8-6a9d-4d16-a08c-7414e6d2c5f8 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:11.886012 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 701/1361] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:31:11.891220 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f48205-98d6-4a19-8545-91c03647ed1f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:11.891670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f48205-98d6-4a19-8545-91c03647ed1f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:11.891793 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f48205-98d6-4a19-8545-91c03647ed1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:11.910034 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-11f48205-98d6-4a19-8545-91c03647ed1f None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:31:11.911245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-11f48205-98d6-4a19-8545-91c03647ed1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:31:11.919110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 661/1362] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:31:11.962695 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af21d8-6a9d-4d16-a08c-7414e6d2c5f8 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:31:11.978153 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af21d8-6a9d-4d16-a08c-7414e6d2c5f8 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:31:11.984579 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be74d9-c4cd-47b0-bfdc-23bb0aa637a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:31:11.985470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be74d9-c4cd-47b0-bfdc-23bb0aa637a6 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:11.985825 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be74d9-c4cd-47b0-bfdc-23bb0aa637a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:11.986171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be74d9-c4cd-47b0-bfdc-23bb0aa637a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:11.995800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be74d9-c4cd-47b0-bfdc-23bb0aa637a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:31:11.997398 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be74d9-c4cd-47b0-bfdc-23bb0aa637a6 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:31:12.007971 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 702/1363] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:31:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:31:14.574145 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e81c0c-c587-4d42-be51-776d79c2ff77 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:14.574629 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e81c0c-c587-4d42-be51-776d79c2ff77 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:14.574920 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e81c0c-c587-4d42-be51-776d79c2ff77 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:14.576201 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 662/1364] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:31:14.581377 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c03172-bd2a-4da5-b03f-6d8479929fae None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:14.581767 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c03172-bd2a-4da5-b03f-6d8479929fae None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:14.581946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c03172-bd2a-4da5-b03f-6d8479929fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:14.600874 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-55c03172-bd2a-4da5-b03f-6d8479929fae None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:31:14.602301 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-55c03172-bd2a-4da5-b03f-6d8479929fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:31:14.611181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 703/1365] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:31:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:31:14.663711 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a4040-7c33-444a-b805-fa87228ee846 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:14.664265 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a4040-7c33-444a-b805-fa87228ee846 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:14.664716 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a4040-7c33-444a-b805-fa87228ee846 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:14.666244 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 663/1366] 10.222.0.67 () {54 vars in 961 bytes} [Tue May 31 14:31:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:31:14.673442 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927de5c4-1f60-4cd8-99b2-6d6d635351b3 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:14.673733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927de5c4-1f60-4cd8-99b2-6d6d635351b3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:14.674045 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927de5c4-1f60-4cd8-99b2-6d6d635351b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:14.696135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-927de5c4-1f60-4cd8-99b2-6d6d635351b3 None None] MFA Rules not processed for user `68c6363b60954c3ebf50d2019f5f2fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:31:14.697258 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-927de5c4-1f60-4cd8-99b2-6d6d635351b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:31:14.706445 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 704/1367] 10.222.0.67 () {58 vars in 1058 bytes} [Tue May 31 14:31:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:31:14.714206 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a4040-7c33-444a-b805-fa87228ee846 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:31:14.728969 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a4040-7c33-444a-b805-fa87228ee846 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:31:14.735234 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5365e-cd1f-40d9-93f8-d0b0d4691883 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:31:14.735941 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5365e-cd1f-40d9-93f8-d0b0d4691883 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:14.736296 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5365e-cd1f-40d9-93f8-d0b0d4691883 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:14.736642 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5365e-cd1f-40d9-93f8-d0b0d4691883 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:14.744981 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5365e-cd1f-40d9-93f8-d0b0d4691883 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:31:14.746555 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5365e-cd1f-40d9-93f8-d0b0d4691883 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:31:14.756701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 664/1368] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:31:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:31:17.083962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35135345-4599-476a-a776-c1930eaabcd3 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:17.084493 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35135345-4599-476a-a776-c1930eaabcd3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:17.084606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35135345-4599-476a-a776-c1930eaabcd3 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:17.086691 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 705/1369] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:31:17.091305 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdca5a-4dcb-43dd-9bfd-eb00ceff748f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:17.091724 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdca5a-4dcb-43dd-9bfd-eb00ceff748f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:17.091867 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdca5a-4dcb-43dd-9bfd-eb00ceff748f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:17.112600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-efcdca5a-4dcb-43dd-9bfd-eb00ceff748f None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:31:17.113821 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-efcdca5a-4dcb-43dd-9bfd-eb00ceff748f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:31:17.124909 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 665/1370] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:31:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:31:17.169952 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35135345-4599-476a-a776-c1930eaabcd3 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:31:17.185539 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35135345-4599-476a-a776-c1930eaabcd3 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:31:17.191089 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413f020-4bb6-4dbc-a45a-af962e571328 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:31:17.191803 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413f020-4bb6-4dbc-a45a-af962e571328 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:17.192160 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413f020-4bb6-4dbc-a45a-af962e571328 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:17.192507 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413f020-4bb6-4dbc-a45a-af962e571328 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:17.201493 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413f020-4bb6-4dbc-a45a-af962e571328 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:31:17.203383 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413f020-4bb6-4dbc-a45a-af962e571328 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:31:17.213484 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 706/1371] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:31:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:31:17.387601 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d88b64-4f1b-4649-842a-9a985e9359be None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:17.388092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d88b64-4f1b-4649-842a-9a985e9359be None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:17.388488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d88b64-4f1b-4649-842a-9a985e9359be None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:17.390281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 666/1372] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:31:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:31:17.396285 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23f4e3-65b5-4cd0-849c-17726c94fabd None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:17.396601 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23f4e3-65b5-4cd0-849c-17726c94fabd None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:17.396892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23f4e3-65b5-4cd0-849c-17726c94fabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:17.416147 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-ad23f4e3-65b5-4cd0-849c-17726c94fabd None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:31:17.417387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ad23f4e3-65b5-4cd0-849c-17726c94fabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:31:17.426395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 707/1373] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:31:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:31:17.741846 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445737f-cffd-42fb-ace7-02b011ca619b None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:17.742201 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445737f-cffd-42fb-ace7-02b011ca619b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:17.742542 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445737f-cffd-42fb-ace7-02b011ca619b None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:17.744127 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 667/1374] 10.222.0.67 () {54 vars in 963 bytes} [Tue May 31 14:31:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:31:17.749013 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83361e4-a43e-4104-89b5-ae0d6bb0077b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:17.750006 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83361e4-a43e-4104-89b5-ae0d6bb0077b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:17.750349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83361e4-a43e-4104-89b5-ae0d6bb0077b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:17.804471 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-f83361e4-a43e-4104-89b5-ae0d6bb0077b None None] MFA Rules not processed for user `990999672d1a4f19896a143e1eeddd44`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:31:17.806302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f83361e4-a43e-4104-89b5-ae0d6bb0077b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:31:17.872468 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 708/1375] 10.222.0.67 () {58 vars in 1060 bytes} [Tue May 31 14:31:17 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:31:17.880336 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445737f-cffd-42fb-ace7-02b011ca619b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:31:17.892425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445737f-cffd-42fb-ace7-02b011ca619b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:31:17.896374 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da268646-1b49-4f82-8599-62175e7e626d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:31:17.897334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da268646-1b49-4f82-8599-62175e7e626d service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:17.897835 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da268646-1b49-4f82-8599-62175e7e626d service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:17.898442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da268646-1b49-4f82-8599-62175e7e626d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:17.908346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da268646-1b49-4f82-8599-62175e7e626d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:31:17.910099 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da268646-1b49-4f82-8599-62175e7e626d service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:31:17.920758 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 668/1376] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:31:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:31:19.498553 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfb1fc-6d79-45f3-9505-199623e7b7cf None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:19.499139 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfb1fc-6d79-45f3-9505-199623e7b7cf None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:19.499139 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfb1fc-6d79-45f3-9505-199623e7b7cf None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:19.501020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 709/1377] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:31:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:31:19.506248 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae5f1a-5310-412d-909f-de18246d47b7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:19.506674 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae5f1a-5310-412d-909f-de18246d47b7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:19.506877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae5f1a-5310-412d-909f-de18246d47b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:19.525314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7cae5f1a-5310-412d-909f-de18246d47b7 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:31:19.526527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7cae5f1a-5310-412d-909f-de18246d47b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:31:19.534545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 669/1378] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:31:19.579570 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfb1fc-6d79-45f3-9505-199623e7b7cf None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:31:19.593710 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfb1fc-6d79-45f3-9505-199623e7b7cf None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:31:19.599414 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b08b66-a174-4654-a50b-1c63fa3b03b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:31:19.600673 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b08b66-a174-4654-a50b-1c63fa3b03b6 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:19.600982 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b08b66-a174-4654-a50b-1c63fa3b03b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:19.601272 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b08b66-a174-4654-a50b-1c63fa3b03b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:19.609909 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b08b66-a174-4654-a50b-1c63fa3b03b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:31:19.611549 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b08b66-a174-4654-a50b-1c63fa3b03b6 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:31:19.615578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:31:19.622469 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 710/1379] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:31:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:31:21.942685 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5686d-09d1-45ec-bbd7-bf4cce3679cb None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:21.943210 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5686d-09d1-45ec-bbd7-bf4cce3679cb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:21.943210 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5686d-09d1-45ec-bbd7-bf4cce3679cb None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:21.944523 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 670/1380] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:31:21.950168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5abea3-9c02-4bed-87dc-17928ae9d437 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:21.950569 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5abea3-9c02-4bed-87dc-17928ae9d437 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:21.950791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5abea3-9c02-4bed-87dc-17928ae9d437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:21.969267 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-4f5abea3-9c02-4bed-87dc-17928ae9d437 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:31:21.970454 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4f5abea3-9c02-4bed-87dc-17928ae9d437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:31:21.978569 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 711/1381] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:31:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:31:22.022728 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5686d-09d1-45ec-bbd7-bf4cce3679cb None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:31:22.039329 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5686d-09d1-45ec-bbd7-bf4cce3679cb None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:31:22.045224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48580044-6b44-40c2-8d72-8e258e517e81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:31:22.045875 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48580044-6b44-40c2-8d72-8e258e517e81 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:22.046163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48580044-6b44-40c2-8d72-8e258e517e81 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:22.046568 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48580044-6b44-40c2-8d72-8e258e517e81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:22.056110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48580044-6b44-40c2-8d72-8e258e517e81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:31:22.057971 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48580044-6b44-40c2-8d72-8e258e517e81 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:31:22.068560 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 671/1382] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:31:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:31:24.400621 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd1354-fd23-4eb6-9a8d-ddfdcdad45ff None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:24.401464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd1354-fd23-4eb6-9a8d-ddfdcdad45ff None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:24.401741 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd1354-fd23-4eb6-9a8d-ddfdcdad45ff None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:24.403154 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 712/1383] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:31:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:31:24.409016 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ed8d9-5094-439f-b930-03058cfe4a95 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:24.409459 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ed8d9-5094-439f-b930-03058cfe4a95 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:24.409574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ed8d9-5094-439f-b930-03058cfe4a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:24.427522 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a01ed8d9-5094-439f-b930-03058cfe4a95 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:31:24.428743 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a01ed8d9-5094-439f-b930-03058cfe4a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:31:24.437052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 672/1384] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:31:24.482892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd1354-fd23-4eb6-9a8d-ddfdcdad45ff None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:31:24.498610 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd1354-fd23-4eb6-9a8d-ddfdcdad45ff None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:31:24.504842 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b303cf-fc61-4b3f-8f13-cec9e4bd6068 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:31:24.505568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b303cf-fc61-4b3f-8f13-cec9e4bd6068 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:24.505923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b303cf-fc61-4b3f-8f13-cec9e4bd6068 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:24.506454 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b303cf-fc61-4b3f-8f13-cec9e4bd6068 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:24.516384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b303cf-fc61-4b3f-8f13-cec9e4bd6068 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:31:24.517948 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b303cf-fc61-4b3f-8f13-cec9e4bd6068 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:31:24.538851 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 713/1385] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:31:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:31:26.882477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62969867-e8c7-4326-bb2f-f2c5ecdc5feb None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:26.883163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62969867-e8c7-4326-bb2f-f2c5ecdc5feb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:26.883163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62969867-e8c7-4326-bb2f-f2c5ecdc5feb None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:26.884960 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 673/1386] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:31:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:31:26.892179 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560352c-7e4d-44f0-b7f7-8e461c8e0ade None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:26.892645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560352c-7e4d-44f0-b7f7-8e461c8e0ade None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:26.892742 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560352c-7e4d-44f0-b7f7-8e461c8e0ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:26.913769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-4560352c-7e4d-44f0-b7f7-8e461c8e0ade None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:31:26.915181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4560352c-7e4d-44f0-b7f7-8e461c8e0ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:31:26.925207 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 714/1387] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:31:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:31:26.969908 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62969867-e8c7-4326-bb2f-f2c5ecdc5feb None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:31:26.989265 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62969867-e8c7-4326-bb2f-f2c5ecdc5feb None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:31:26.996236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a1420-3a03-4de2-a2d6-ac839c6d638a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:31:26.996901 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a1420-3a03-4de2-a2d6-ac839c6d638a service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:26.997191 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a1420-3a03-4de2-a2d6-ac839c6d638a service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:26.997474 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a1420-3a03-4de2-a2d6-ac839c6d638a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:27.009146 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a1420-3a03-4de2-a2d6-ac839c6d638a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:31:27.010769 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a1420-3a03-4de2-a2d6-ac839c6d638a service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:31:27.021472 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 674/1388] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:31:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:31:29.565449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee4ae9-1ff2-48f2-ac34-491d7cecd831 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:29.566107 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee4ae9-1ff2-48f2-ac34-491d7cecd831 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:29.566107 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee4ae9-1ff2-48f2-ac34-491d7cecd831 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:29.567350 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 715/1389] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:31:29.573325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdcb0b-8236-49a0-b928-b9a60aa172b0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:29.573807 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdcb0b-8236-49a0-b928-b9a60aa172b0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:29.573972 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdcb0b-8236-49a0-b928-b9a60aa172b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:29.593739 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a2cdcb0b-8236-49a0-b928-b9a60aa172b0 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:31:29.595215 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a2cdcb0b-8236-49a0-b928-b9a60aa172b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:31:29.604362 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 675/1390] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:31:29.656028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee4ae9-1ff2-48f2-ac34-491d7cecd831 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:31:29.672345 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee4ae9-1ff2-48f2-ac34-491d7cecd831 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:31:29.678623 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f6acd-2006-4a37-92ba-9caa6945857b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:31:29.679393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f6acd-2006-4a37-92ba-9caa6945857b service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:29.679740 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f6acd-2006-4a37-92ba-9caa6945857b service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:29.680076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f6acd-2006-4a37-92ba-9caa6945857b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:29.690506 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f6acd-2006-4a37-92ba-9caa6945857b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:31:29.692428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f6acd-2006-4a37-92ba-9caa6945857b service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:31:29.704040 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 716/1391] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:31:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:31:32.057857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06071fa4-f783-448c-a60a-fa8122de8fdb None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:32.058514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06071fa4-f783-448c-a60a-fa8122de8fdb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:32.058514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06071fa4-f783-448c-a60a-fa8122de8fdb None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:32.059819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 676/1392] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:31:32.065549 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30e81a-395d-41ce-a178-7a961fd57fe9 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:32.065964 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30e81a-395d-41ce-a178-7a961fd57fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:32.066111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30e81a-395d-41ce-a178-7a961fd57fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:32.084104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-9a30e81a-395d-41ce-a178-7a961fd57fe9 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:31:32.085311 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9a30e81a-395d-41ce-a178-7a961fd57fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:31:32.093635 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 717/1393] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:31:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:31:32.138956 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06071fa4-f783-448c-a60a-fa8122de8fdb None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:31:32.153528 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06071fa4-f783-448c-a60a-fa8122de8fdb None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:31:32.159799 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af14fd-fad9-4723-a3e4-5c47c2598b46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:31:32.160449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af14fd-fad9-4723-a3e4-5c47c2598b46 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:32.160736 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af14fd-fad9-4723-a3e4-5c47c2598b46 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:32.161012 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af14fd-fad9-4723-a3e4-5c47c2598b46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:32.169501 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af14fd-fad9-4723-a3e4-5c47c2598b46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:31:32.171655 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af14fd-fad9-4723-a3e4-5c47c2598b46 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:31:32.181219 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 677/1394] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:31:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:31:33.864085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88bc2c-3774-4347-8963-9175efa23828 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:33.864655 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88bc2c-3774-4347-8963-9175efa23828 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:33.864655 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88bc2c-3774-4347-8963-9175efa23828 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:33.866243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 718/1395] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:31:33.871224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6b7ed-2fca-4450-9a65-8ed19e24a9b9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:33.871626 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6b7ed-2fca-4450-9a65-8ed19e24a9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:33.871790 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6b7ed-2fca-4450-9a65-8ed19e24a9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:33.878065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=52 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:31:33.890696 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-71b6b7ed-2fca-4450-9a65-8ed19e24a9b9 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:31:33.891899 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-71b6b7ed-2fca-4450-9a65-8ed19e24a9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:31:33.899747 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 678/1396] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:31:33.956317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a09b7-b1b8-4e15-926f-bd8c0ed3b24d None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:33.956614 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a09b7-b1b8-4e15-926f-bd8c0ed3b24d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:33.956894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a09b7-b1b8-4e15-926f-bd8c0ed3b24d None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:33.958262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 719/1397] 10.222.0.67 () {54 vars in 962 bytes} [Tue May 31 14:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:31:33.963249 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf32ec-c19b-4e16-99ad-0d3d97862066 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:33.963606 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf32ec-c19b-4e16-99ad-0d3d97862066 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:33.963946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf32ec-c19b-4e16-99ad-0d3d97862066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:34.012051 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-76bf32ec-c19b-4e16-99ad-0d3d97862066 None None] MFA Rules not processed for user `d8a4b0dd4e21414d91fe7bed478d15e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:31:34.013287 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-76bf32ec-c19b-4e16-99ad-0d3d97862066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:31:34.073721 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 679/1398] 10.222.0.67 () {58 vars in 1059 bytes} [Tue May 31 14:31:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:31:34.081677 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a09b7-b1b8-4e15-926f-bd8c0ed3b24d None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:31:34.092366 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a09b7-b1b8-4e15-926f-bd8c0ed3b24d None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:31:34.096727 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cd774-6c20-437a-b2ec-304f037c42db service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8a4b0dd4e21414d91fe7bed478d15e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:31:34.097686 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cd774-6c20-437a-b2ec-304f037c42db service glance] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:34.098040 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cd774-6c20-437a-b2ec-304f037c42db service glance] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:34.098323 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cd774-6c20-437a-b2ec-304f037c42db service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:34.107544 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21cd774-6c20-437a-b2ec-304f037c42db service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:31:34.108978 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21cd774-6c20-437a-b2ec-304f037c42db service glance] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:31:34.118862 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 720/1399] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:31:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:31:34.500624 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87293b4-9f9d-441c-b014-c3cff71a5933 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:34.500921 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87293b4-9f9d-441c-b014-c3cff71a5933 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:34.501210 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87293b4-9f9d-441c-b014-c3cff71a5933 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:34.503180 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 680/1400] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:31:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 31 14:31:34.508792 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddee800-d067-42a1-9ec1-014530b18e40 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:34.509083 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddee800-d067-42a1-9ec1-014530b18e40 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:34.509366 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddee800-d067-42a1-9ec1-014530b18e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:34.527409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5ddee800-d067-42a1-9ec1-014530b18e40 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:31:34.528643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5ddee800-d067-42a1-9ec1-014530b18e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:31:34.536863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 721/1401] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:31:34.581277 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87293b4-9f9d-441c-b014-c3cff71a5933 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:31:34.596224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87293b4-9f9d-441c-b014-c3cff71a5933 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:31:34.602074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052b084-a31d-4e05-ae8b-e5d538ace809 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:31:34.602821 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052b084-a31d-4e05-ae8b-e5d538ace809 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:34.603106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052b084-a31d-4e05-ae8b-e5d538ace809 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:34.603383 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052b084-a31d-4e05-ae8b-e5d538ace809 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:34.611722 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a052b084-a31d-4e05-ae8b-e5d538ace809 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:31:34.613294 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a052b084-a31d-4e05-ae8b-e5d538ace809 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:31:34.622869 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 681/1402] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:31:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:31:34.651070 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7b185-b242-445a-ae56-e8a6c70e2609 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:34.651363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7b185-b242-445a-ae56-e8a6c70e2609 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:34.651639 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7b185-b242-445a-ae56-e8a6c70e2609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:34.715364 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-0fa7b185-b242-445a-ae56-e8a6c70e2609 None None] MFA Rules not processed for user `0b7d100f2407432381d119c25814fff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:31:34.716760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0fa7b185-b242-445a-ae56-e8a6c70e2609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:31:34.787072 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 722/1403] 10.222.0.67 () {58 vars in 1009 bytes} [Tue May 31 14:31:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:31:34.834278 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845a4c2-e51a-44ef-8878-0ed345d1a80e None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:34.834573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845a4c2-e51a-44ef-8878-0ed345d1a80e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:34.834920 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845a4c2-e51a-44ef-8878-0ed345d1a80e None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:34.836340 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 682/1404] 10.222.0.67 () {54 vars in 942 bytes} [Tue May 31 14:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:31:34.843517 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4f8a6-f247-4ad2-a890-720b0465440d None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:34.843881 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4f8a6-f247-4ad2-a890-720b0465440d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:34.844164 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4f8a6-f247-4ad2-a890-720b0465440d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:34.896202 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-4ee4f8a6-f247-4ad2-a890-720b0465440d None None] MFA Rules not processed for user `dfa2282f957941b98db1d7497da3351b`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:31:34.897330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4ee4f8a6-f247-4ad2-a890-720b0465440d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:31:34.931254 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:31:34.970229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 723/1405] 10.222.0.67 () {58 vars in 1039 bytes} [Tue May 31 14:31:34 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:31:34.979452 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845a4c2-e51a-44ef-8878-0ed345d1a80e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:31:34.993317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845a4c2-e51a-44ef-8878-0ed345d1a80e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:31:34.998303 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a5e7f-8e83-445e-b9a2-7af6c00e6395 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfa2282f957941b98db1d7497da3351b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:31:34.999591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a5e7f-8e83-445e-b9a2-7af6c00e6395 service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:34.999883 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a5e7f-8e83-445e-b9a2-7af6c00e6395 service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:35.000161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a5e7f-8e83-445e-b9a2-7af6c00e6395 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:35.008251 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a5e7f-8e83-445e-b9a2-7af6c00e6395 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:31:35.009812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a5e7f-8e83-445e-b9a2-7af6c00e6395 service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:31:35.018076 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 683/1406] 10.222.0.67 () {60 vars in 1384 bytes} [Tue May 31 14:31:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 31 14:31:37.129772 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a383a-c7c7-417c-96c1-19dbcf26c24c None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:37.130321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a383a-c7c7-417c-96c1-19dbcf26c24c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:37.130321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a383a-c7c7-417c-96c1-19dbcf26c24c None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:37.133403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 724/1407] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:31:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:31:37.138710 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0273fd-b107-4277-abf7-34de4e947ac4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:37.139117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0273fd-b107-4277-abf7-34de4e947ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:37.139305 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0273fd-b107-4277-abf7-34de4e947ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:37.158946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-fc0273fd-b107-4277-abf7-34de4e947ac4 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:31:37.160494 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fc0273fd-b107-4277-abf7-34de4e947ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:31:37.168659 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 684/1408] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:31:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:31:37.214693 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a383a-c7c7-417c-96c1-19dbcf26c24c None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:31:37.229313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a383a-c7c7-417c-96c1-19dbcf26c24c None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:31:37.235397 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0728b1c-161e-4676-a314-f0247bfcf9f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:31:37.236123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0728b1c-161e-4676-a314-f0247bfcf9f0 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:37.236484 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0728b1c-161e-4676-a314-f0247bfcf9f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:37.236767 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0728b1c-161e-4676-a314-f0247bfcf9f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:37.246069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0728b1c-161e-4676-a314-f0247bfcf9f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:31:37.247709 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0728b1c-161e-4676-a314-f0247bfcf9f0 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:31:37.259961 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 725/1409] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:31:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:31:47.151790 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=55 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:31:47.253953 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=21 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:31:58.271057 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b8947-ce62-4c9e-8532-da7175abdf4a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:58.271622 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b8947-ce62-4c9e-8532-da7175abdf4a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:58.271622 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b8947-ce62-4c9e-8532-da7175abdf4a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:58.272987 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 685/1410] 10.222.0.67 () {54 vars in 918 bytes} [Tue May 31 14:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:31:58.278230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09717a7-ee7f-48b4-805b-f044e8151b6b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:58.278629 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09717a7-ee7f-48b4-805b-f044e8151b6b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:58.278844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09717a7-ee7f-48b4-805b-f044e8151b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:58.297356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a09717a7-ee7f-48b4-805b-f044e8151b6b None None] MFA Rules not processed for user `6c1c13515a8b41a781478dc65e6a88b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:31:58.298561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a09717a7-ee7f-48b4-805b-f044e8151b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:31:58.307102 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 726/1411] 10.222.0.67 () {58 vars in 1015 bytes} [Tue May 31 14:31:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:31:58.321806 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b7a27-353a-4bf1-8954-57726864071b None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:58.322100 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b7a27-353a-4bf1-8954-57726864071b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:58.322382 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b7a27-353a-4bf1-8954-57726864071b None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:58.323767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 686/1412] 10.222.0.67 () {54 vars in 918 bytes} [Tue May 31 14:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:31:58.328134 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a4543-7d93-45dd-b70f-b4f0ff9cb7fc None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:31:58.328420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a4543-7d93-45dd-b70f-b4f0ff9cb7fc None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:31:58.328703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a4543-7d93-45dd-b70f-b4f0ff9cb7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:31:58.346325 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-cb6a4543-7d93-45dd-b70f-b4f0ff9cb7fc None None] MFA Rules not processed for user `68c6363b60954c3ebf50d2019f5f2fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:31:58.347539 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cb6a4543-7d93-45dd-b70f-b4f0ff9cb7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:31:58.356499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 727/1413] 10.222.0.67 () {58 vars in 1015 bytes} [Tue May 31 14:31:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:32:01.046881 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015c6ce-d4a0-497d-86d5-05bdd311ff2e None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:01.047418 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015c6ce-d4a0-497d-86d5-05bdd311ff2e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:01.047418 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015c6ce-d4a0-497d-86d5-05bdd311ff2e None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:01.048802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 687/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 31 14:32:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:32:01.054143 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d208845-0667-4a55-a1c2-3b55cbfa5423 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:01.054608 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d208845-0667-4a55-a1c2-3b55cbfa5423 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:01.054762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d208845-0667-4a55-a1c2-3b55cbfa5423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:01.073056 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-9d208845-0667-4a55-a1c2-3b55cbfa5423 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:32:01.074248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9d208845-0667-4a55-a1c2-3b55cbfa5423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:32:01.082189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 728/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 31 14:32:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:32:01.125545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015c6ce-d4a0-497d-86d5-05bdd311ff2e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:32:01.141100 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015c6ce-d4a0-497d-86d5-05bdd311ff2e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:32:01.146526 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee1172-3368-4e12-b877-88555137d93c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:32:01.147198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee1172-3368-4e12-b877-88555137d93c service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:01.147485 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee1172-3368-4e12-b877-88555137d93c service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:01.147789 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee1172-3368-4e12-b877-88555137d93c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:01.156368 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee1172-3368-4e12-b877-88555137d93c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:32:01.158475 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee1172-3368-4e12-b877-88555137d93c service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:32:01.169199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 688/1416] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:32:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:32:02.142074 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d057f2e-5e35-4080-888f-df20d8c273e5 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:02.142553 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d057f2e-5e35-4080-888f-df20d8c273e5 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:02.142672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d057f2e-5e35-4080-888f-df20d8c273e5 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:02.144042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 729/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 31 14:32:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 31 14:32:02.148920 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440a9b6-a2ac-4f21-8a49-c6aa835d04e9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:02.149344 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440a9b6-a2ac-4f21-8a49-c6aa835d04e9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:02.149483 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440a9b6-a2ac-4f21-8a49-c6aa835d04e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:02.167220 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b440a9b6-a2ac-4f21-8a49-c6aa835d04e9 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:32:02.168412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b440a9b6-a2ac-4f21-8a49-c6aa835d04e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:32:02.176157 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 689/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 31 14:32:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:32:02.341403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d057f2e-5e35-4080-888f-df20d8c273e5 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:32:02.355640 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d057f2e-5e35-4080-888f-df20d8c273e5 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:32:02.361006 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30eb2a-1d7e-4c4d-b427-12bdebdfd251 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:32:02.361640 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30eb2a-1d7e-4c4d-b427-12bdebdfd251 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:02.361928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30eb2a-1d7e-4c4d-b427-12bdebdfd251 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:02.362209 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30eb2a-1d7e-4c4d-b427-12bdebdfd251 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:02.370196 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30eb2a-1d7e-4c4d-b427-12bdebdfd251 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:32:02.371819 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30eb2a-1d7e-4c4d-b427-12bdebdfd251 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:32:02.381686 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 730/1419] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:32:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:32:03.330319 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e7aae-8da7-49b5-9390-8c3512461d34 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:03.330880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e7aae-8da7-49b5-9390-8c3512461d34 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:03.330880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e7aae-8da7-49b5-9390-8c3512461d34 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:03.332283 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 690/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 31 14:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:32:03.337171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e18a57-9f70-4684-90c1-34b6800f28d1 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:03.337656 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e18a57-9f70-4684-90c1-34b6800f28d1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:03.337755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e18a57-9f70-4684-90c1-34b6800f28d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:03.356318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-17e18a57-9f70-4684-90c1-34b6800f28d1 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:32:03.357520 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-17e18a57-9f70-4684-90c1-34b6800f28d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:32:03.365761 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 731/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 31 14:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:32:03.515752 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e7aae-8da7-49b5-9390-8c3512461d34 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:32:03.530230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e7aae-8da7-49b5-9390-8c3512461d34 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:32:03.535556 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f0ef6-73e6-41ca-8650-f666abc762c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:32:03.536199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f0ef6-73e6-41ca-8650-f666abc762c2 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:03.536485 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f0ef6-73e6-41ca-8650-f666abc762c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:03.536766 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f0ef6-73e6-41ca-8650-f666abc762c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:03.544778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f0ef6-73e6-41ca-8650-f666abc762c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:32:03.546868 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f0ef6-73e6-41ca-8650-f666abc762c2 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:32:03.556250 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 691/1422] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:32:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:32:04.496074 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b7bf4-d211-4379-b858-f1e319ce7441 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:04.496612 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b7bf4-d211-4379-b858-f1e319ce7441 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:04.496612 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b7bf4-d211-4379-b858-f1e319ce7441 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:04.497911 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 732/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 31 14:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:32:04.503446 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb16941-827f-4d64-9c1f-a049514a15b5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:04.503894 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb16941-827f-4d64-9c1f-a049514a15b5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:04.503990 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb16941-827f-4d64-9c1f-a049514a15b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:04.523585 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2fb16941-827f-4d64-9c1f-a049514a15b5 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:32:04.524795 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2fb16941-827f-4d64-9c1f-a049514a15b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:32:04.532863 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 692/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 31 14:32:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:32:04.680559 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b7bf4-d211-4379-b858-f1e319ce7441 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:32:04.696956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b7bf4-d211-4379-b858-f1e319ce7441 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:32:04.703323 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d13dab-7c7b-4848-8e11-c73834c7d06c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:32:04.703957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d13dab-7c7b-4848-8e11-c73834c7d06c service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:04.704237 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d13dab-7c7b-4848-8e11-c73834c7d06c service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:04.704510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d13dab-7c7b-4848-8e11-c73834c7d06c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:04.712644 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d13dab-7c7b-4848-8e11-c73834c7d06c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:32:04.714207 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d13dab-7c7b-4848-8e11-c73834c7d06c service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:32:04.723471 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 733/1425] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:32:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:32:05.626764 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1a5a6-7e32-4752-b924-8982103d642c None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:05.627304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1a5a6-7e32-4752-b924-8982103d642c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:05.627416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1a5a6-7e32-4752-b924-8982103d642c None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:05.628726 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 693/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 31 14:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:32:05.633779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae3f70-ca30-47fc-a493-59f446196224 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:05.634205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae3f70-ca30-47fc-a493-59f446196224 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:05.634369 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae3f70-ca30-47fc-a493-59f446196224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:05.652558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a1ae3f70-ca30-47fc-a493-59f446196224 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:32:05.653764 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a1ae3f70-ca30-47fc-a493-59f446196224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:32:05.662229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 734/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 31 14:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:32:05.811390 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1a5a6-7e32-4752-b924-8982103d642c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:32:05.826287 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1a5a6-7e32-4752-b924-8982103d642c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:32:05.832524 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290830c-4b26-4bf5-886e-93760d1ea6b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:32:05.833176 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290830c-4b26-4bf5-886e-93760d1ea6b9 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:05.833455 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290830c-4b26-4bf5-886e-93760d1ea6b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:05.833730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290830c-4b26-4bf5-886e-93760d1ea6b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:05.842011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290830c-4b26-4bf5-886e-93760d1ea6b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:32:05.844241 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290830c-4b26-4bf5-886e-93760d1ea6b9 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:32:05.853556 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 694/1428] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:32:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:32:06.700600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f9331-957e-478a-9cac-696a0cea1d79 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:06.701134 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f9331-957e-478a-9cac-696a0cea1d79 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:06.701134 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f9331-957e-478a-9cac-696a0cea1d79 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:06.702556 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 735/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 31 14:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:32:06.707587 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072752e-2fb2-46ba-bab6-df056b3f7342 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:06.707976 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072752e-2fb2-46ba-bab6-df056b3f7342 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:06.708147 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072752e-2fb2-46ba-bab6-df056b3f7342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:06.726285 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b072752e-2fb2-46ba-bab6-df056b3f7342 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:32:06.727518 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b072752e-2fb2-46ba-bab6-df056b3f7342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:32:06.735825 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 695/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 31 14:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:32:06.892796 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f9331-957e-478a-9cac-696a0cea1d79 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:32:06.907162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f9331-957e-478a-9cac-696a0cea1d79 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:32:06.912422 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd3f0e2-3f86-4cb1-8b1d-646428ad0903 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:32:06.913053 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3f0e2-3f86-4cb1-8b1d-646428ad0903 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:06.913332 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3f0e2-3f86-4cb1-8b1d-646428ad0903 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:06.913606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3f0e2-3f86-4cb1-8b1d-646428ad0903 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:06.922124 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd3f0e2-3f86-4cb1-8b1d-646428ad0903 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:32:06.923748 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd3f0e2-3f86-4cb1-8b1d-646428ad0903 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:32:06.932860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 736/1431] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:32:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:32:07.794998 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc0d96-381a-4a07-89d9-6ae63de887a1 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:07.795599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc0d96-381a-4a07-89d9-6ae63de887a1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:07.795599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc0d96-381a-4a07-89d9-6ae63de887a1 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:07.797015 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 696/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 31 14:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:32:07.803292 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a0fe3-2c0d-47bd-ad58-489bc42ae860 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:07.803762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a0fe3-2c0d-47bd-ad58-489bc42ae860 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:07.803859 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a0fe3-2c0d-47bd-ad58-489bc42ae860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:07.824020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-548a0fe3-2c0d-47bd-ad58-489bc42ae860 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:32:07.825233 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-548a0fe3-2c0d-47bd-ad58-489bc42ae860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:32:07.833560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 737/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 31 14:32:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:32:07.981414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc0d96-381a-4a07-89d9-6ae63de887a1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:32:07.995529 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc0d96-381a-4a07-89d9-6ae63de887a1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:32:08.000929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7155700-7d9d-4219-a098-e982613a99d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:32:08.001567 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7155700-7d9d-4219-a098-e982613a99d1 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:08.001857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7155700-7d9d-4219-a098-e982613a99d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:08.002136 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7155700-7d9d-4219-a098-e982613a99d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:08.010247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7155700-7d9d-4219-a098-e982613a99d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:32:08.011865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7155700-7d9d-4219-a098-e982613a99d1 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:32:08.021604 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 697/1434] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:32:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:32:08.877553 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be2caa-90c0-4462-94bb-42708b7a39ca None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:08.878110 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be2caa-90c0-4462-94bb-42708b7a39ca None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:08.878110 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be2caa-90c0-4462-94bb-42708b7a39ca None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:08.879539 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 738/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 31 14:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:32:08.884544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da47303-98f9-4779-96f5-e9e627a420d1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:08.884925 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da47303-98f9-4779-96f5-e9e627a420d1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:08.885110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da47303-98f9-4779-96f5-e9e627a420d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:08.903717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7da47303-98f9-4779-96f5-e9e627a420d1 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:32:08.904924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7da47303-98f9-4779-96f5-e9e627a420d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:32:08.913441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 698/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 31 14:32:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:32:09.063089 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be2caa-90c0-4462-94bb-42708b7a39ca None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:32:09.077713 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be2caa-90c0-4462-94bb-42708b7a39ca None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:32:09.083006 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d7971-2b21-4dda-8930-c5418b7d5a74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:32:09.083637 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d7971-2b21-4dda-8930-c5418b7d5a74 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:09.083918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d7971-2b21-4dda-8930-c5418b7d5a74 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:09.084193 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d7971-2b21-4dda-8930-c5418b7d5a74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:09.092358 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d7971-2b21-4dda-8930-c5418b7d5a74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:32:09.094449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d7971-2b21-4dda-8930-c5418b7d5a74 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:32:09.103927 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 739/1437] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:32:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:32:09.958879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e7290-d70b-4b0d-873a-5a103202b041 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:09.959432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e7290-d70b-4b0d-873a-5a103202b041 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:09.959432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e7290-d70b-4b0d-873a-5a103202b041 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:09.960827 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 699/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 31 14:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:32:09.965656 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d1a90-38f3-4188-8944-144fb385221f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:09.966079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d1a90-38f3-4188-8944-144fb385221f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:09.966198 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d1a90-38f3-4188-8944-144fb385221f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:09.984877 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-e35d1a90-38f3-4188-8944-144fb385221f None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:32:09.986075 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e35d1a90-38f3-4188-8944-144fb385221f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:32:09.993822 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 740/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 31 14:32:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:32:10.142146 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e7290-d70b-4b0d-873a-5a103202b041 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:32:10.156347 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e7290-d70b-4b0d-873a-5a103202b041 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:32:10.161754 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b80a52-6a6c-4627-96ca-71c716097cbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:32:10.162400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b80a52-6a6c-4627-96ca-71c716097cbb service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:10.162727 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b80a52-6a6c-4627-96ca-71c716097cbb service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:10.163016 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b80a52-6a6c-4627-96ca-71c716097cbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:10.171602 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b80a52-6a6c-4627-96ca-71c716097cbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:32:10.173192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b80a52-6a6c-4627-96ca-71c716097cbb service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:32:10.182533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 700/1440] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:32:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:32:12.266162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7926d-f496-4bf0-b974-0c3f45d999eb None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:12.266735 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7926d-f496-4bf0-b974-0c3f45d999eb None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:12.266937 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7926d-f496-4bf0-b974-0c3f45d999eb None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:12.268151 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 741/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 31 14:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:32:12.273597 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fa0c0-3bcc-482f-a788-4e7b58e9f3ec None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:12.274036 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fa0c0-3bcc-482f-a788-4e7b58e9f3ec None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:12.274158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fa0c0-3bcc-482f-a788-4e7b58e9f3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:12.292166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b55fa0c0-3bcc-482f-a788-4e7b58e9f3ec None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:32:12.293366 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b55fa0c0-3bcc-482f-a788-4e7b58e9f3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:32:12.301780 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 701/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 31 14:32:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:32:12.340844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7926d-f496-4bf0-b974-0c3f45d999eb None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:32:12.351584 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7926d-f496-4bf0-b974-0c3f45d999eb None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:32:12.355161 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab231e20-478b-4cd5-81e5-d174ae4ac283 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:32:12.355798 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab231e20-478b-4cd5-81e5-d174ae4ac283 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:12.356087 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab231e20-478b-4cd5-81e5-d174ae4ac283 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:12.356367 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab231e20-478b-4cd5-81e5-d174ae4ac283 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:12.365688 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab231e20-478b-4cd5-81e5-d174ae4ac283 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:32:12.367186 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab231e20-478b-4cd5-81e5-d174ae4ac283 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:32:12.377141 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 742/1443] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:32:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:32:14.759198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828ea62-efba-4c0f-9a8f-ba59fae38a3c None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:14.759779 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828ea62-efba-4c0f-9a8f-ba59fae38a3c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:14.759779 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828ea62-efba-4c0f-9a8f-ba59fae38a3c None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:14.761127 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 702/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 31 14:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:32:14.767139 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb292e4d-caed-46ec-b285-1521e290b630 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:14.767555 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb292e4d-caed-46ec-b285-1521e290b630 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:14.767690 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb292e4d-caed-46ec-b285-1521e290b630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:14.785685 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-bb292e4d-caed-46ec-b285-1521e290b630 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:32:14.786905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-bb292e4d-caed-46ec-b285-1521e290b630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:32:14.794963 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 743/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 31 14:32:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:32:14.833021 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828ea62-efba-4c0f-9a8f-ba59fae38a3c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:32:14.844321 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828ea62-efba-4c0f-9a8f-ba59fae38a3c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:32:14.847818 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088e7c7-729a-406e-b18a-447a25f0964b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:32:14.848451 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088e7c7-729a-406e-b18a-447a25f0964b service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:14.848737 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088e7c7-729a-406e-b18a-447a25f0964b service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:14.849013 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088e7c7-729a-406e-b18a-447a25f0964b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:14.857797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f088e7c7-729a-406e-b18a-447a25f0964b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:32:14.859246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f088e7c7-729a-406e-b18a-447a25f0964b service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:32:14.868365 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 703/1446] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:32:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:32:17.474925 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105173ed-88c2-40c5-8431-984ba58b4987 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:17.475433 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105173ed-88c2-40c5-8431-984ba58b4987 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:17.475433 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105173ed-88c2-40c5-8431-984ba58b4987 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:17.476776 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 744/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 31 14:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:32:17.482101 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067cc14-fa34-4c04-be9c-5847a627a1de None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:17.482508 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067cc14-fa34-4c04-be9c-5847a627a1de None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:17.482673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067cc14-fa34-4c04-be9c-5847a627a1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:17.501149 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4067cc14-fa34-4c04-be9c-5847a627a1de None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:32:17.502341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4067cc14-fa34-4c04-be9c-5847a627a1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:32:17.510041 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 704/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 31 14:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:32:17.548185 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105173ed-88c2-40c5-8431-984ba58b4987 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:32:17.558826 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105173ed-88c2-40c5-8431-984ba58b4987 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:32:17.562906 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae2baa-f641-447f-919f-81f37c7233f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:32:17.563535 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae2baa-f641-447f-919f-81f37c7233f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:17.563819 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae2baa-f641-447f-919f-81f37c7233f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:17.564094 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae2baa-f641-447f-919f-81f37c7233f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:17.573660 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae2baa-f641-447f-919f-81f37c7233f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:32:17.575120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae2baa-f641-447f-919f-81f37c7233f0 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:32:17.584333 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 745/1449] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:32:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:32:19.957215 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3da08-04aa-424d-8f8a-4d6c2d07dfc5 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:19.957834 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3da08-04aa-424d-8f8a-4d6c2d07dfc5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:19.957834 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3da08-04aa-424d-8f8a-4d6c2d07dfc5 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:19.959259 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 705/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 31 14:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:32:19.964566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99059d38-4570-47fc-8bbd-73921b112340 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:19.965016 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99059d38-4570-47fc-8bbd-73921b112340 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:19.965126 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99059d38-4570-47fc-8bbd-73921b112340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:19.982873 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-99059d38-4570-47fc-8bbd-73921b112340 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:32:19.984620 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-99059d38-4570-47fc-8bbd-73921b112340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:32:19.993499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 746/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 31 14:32:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:32:20.035920 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3da08-04aa-424d-8f8a-4d6c2d07dfc5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:32:20.037612 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f9e3da08-04aa-424d-8f8a-4d6c2d07dfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:32:20.050959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3da08-04aa-424d-8f8a-4d6c2d07dfc5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:32:20.054456 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbaa649-ccb2-49d6-98b1-61fb3288c40c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:32:20.055162 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbaa649-ccb2-49d6-98b1-61fb3288c40c service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:20.055454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbaa649-ccb2-49d6-98b1-61fb3288c40c service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:20.055735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbaa649-ccb2-49d6-98b1-61fb3288c40c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:20.065138 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbaa649-ccb2-49d6-98b1-61fb3288c40c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:32:20.066556 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbaa649-ccb2-49d6-98b1-61fb3288c40c service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:32:20.076346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 706/1452] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:32:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:32:22.423677 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f16d4-5ee8-412d-b3ac-868faba0ed2d None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:22.424203 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f16d4-5ee8-412d-b3ac-868faba0ed2d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:22.424203 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f16d4-5ee8-412d-b3ac-868faba0ed2d None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:22.425537 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 747/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 31 14:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:32:22.430957 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296876a9-bdf5-49e7-b125-9818affeef0b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:22.431401 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296876a9-bdf5-49e7-b125-9818affeef0b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:22.431531 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296876a9-bdf5-49e7-b125-9818affeef0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:22.449788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-296876a9-bdf5-49e7-b125-9818affeef0b None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:32:22.451018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-296876a9-bdf5-49e7-b125-9818affeef0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:32:22.459718 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 707/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 31 14:32:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:32:22.496900 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f16d4-5ee8-412d-b3ac-868faba0ed2d None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:32:22.507942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f16d4-5ee8-412d-b3ac-868faba0ed2d None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:32:22.511421 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94df463-dc07-4063-bf64-eb61925ebff0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:32:22.512072 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94df463-dc07-4063-bf64-eb61925ebff0 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:22.512355 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94df463-dc07-4063-bf64-eb61925ebff0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:22.512627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94df463-dc07-4063-bf64-eb61925ebff0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:22.522257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94df463-dc07-4063-bf64-eb61925ebff0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:32:22.523711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94df463-dc07-4063-bf64-eb61925ebff0 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:32:22.533198 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 748/1455] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:32:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:32:24.849086 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81991288-74d6-4bd9-bf99-78c289cfcfec None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:24.849623 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81991288-74d6-4bd9-bf99-78c289cfcfec None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:24.849623 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81991288-74d6-4bd9-bf99-78c289cfcfec None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:24.851013 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 708/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 31 14:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:32:24.856352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb751d-d948-444c-bd00-bf4b144ba230 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:24.856782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb751d-d948-444c-bd00-bf4b144ba230 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:24.856900 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb751d-d948-444c-bd00-bf4b144ba230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:24.875011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-0ddb751d-d948-444c-bd00-bf4b144ba230 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:32:24.876240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0ddb751d-d948-444c-bd00-bf4b144ba230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:32:24.883928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 749/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 31 14:32:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:32:24.922265 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81991288-74d6-4bd9-bf99-78c289cfcfec None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:32:24.933008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81991288-74d6-4bd9-bf99-78c289cfcfec None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:32:24.936617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e835a2-4aae-4997-8333-6e1604be1818 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:32:24.937253 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e835a2-4aae-4997-8333-6e1604be1818 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:24.937541 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e835a2-4aae-4997-8333-6e1604be1818 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:24.937825 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e835a2-4aae-4997-8333-6e1604be1818 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:24.947621 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e835a2-4aae-4997-8333-6e1604be1818 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:32:24.949053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e835a2-4aae-4997-8333-6e1604be1818 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:32:24.958935 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 709/1458] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:32:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:32:27.232407 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32bd9c-19c1-4a50-993b-d7f0f5502bcc None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:27.232924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32bd9c-19c1-4a50-993b-d7f0f5502bcc None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:27.232924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32bd9c-19c1-4a50-993b-d7f0f5502bcc None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:27.234769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 750/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 31 14:32:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:32:27.239987 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82413cc-904f-4070-ac05-b8c5a42182c8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:27.240425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82413cc-904f-4070-ac05-b8c5a42182c8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:27.240538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82413cc-904f-4070-ac05-b8c5a42182c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:27.246817 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=70 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:32:27.262784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a82413cc-904f-4070-ac05-b8c5a42182c8 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:32:27.264067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a82413cc-904f-4070-ac05-b8c5a42182c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:32:27.272225 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 710/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 31 14:32:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:32:27.309881 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32bd9c-19c1-4a50-993b-d7f0f5502bcc None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:32:27.316182 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=73 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:32:27.322615 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32bd9c-19c1-4a50-993b-d7f0f5502bcc None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:32:27.326278 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d1f6c-a80f-45ab-824e-60a35b2edec8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:32:27.326939 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d1f6c-a80f-45ab-824e-60a35b2edec8 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:27.327227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d1f6c-a80f-45ab-824e-60a35b2edec8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:27.327507 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d1f6c-a80f-45ab-824e-60a35b2edec8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:27.336342 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d1f6c-a80f-45ab-824e-60a35b2edec8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:32:27.337766 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d1f6c-a80f-45ab-824e-60a35b2edec8 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:32:27.347106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 751/1461] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:32:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:32:29.611435 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f727e-6a0f-4a4d-8d8c-9f2ca1e51213 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:29.612007 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f727e-6a0f-4a4d-8d8c-9f2ca1e51213 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:29.612007 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f727e-6a0f-4a4d-8d8c-9f2ca1e51213 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:29.613341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 711/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 31 14:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:32:29.618588 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcc367-d07d-4c8f-bd8e-0afaeae418f8 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:29.619069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcc367-d07d-4c8f-bd8e-0afaeae418f8 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:29.619185 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcc367-d07d-4c8f-bd8e-0afaeae418f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:29.636647 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-ebdcc367-d07d-4c8f-bd8e-0afaeae418f8 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:32:29.637844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ebdcc367-d07d-4c8f-bd8e-0afaeae418f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:32:29.646257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 752/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 31 14:32:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:32:29.683666 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f727e-6a0f-4a4d-8d8c-9f2ca1e51213 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:32:29.694694 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f727e-6a0f-4a4d-8d8c-9f2ca1e51213 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:32:29.698393 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e165812-fdac-494a-9fc6-cb3dcff0cf1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:32:29.699058 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e165812-fdac-494a-9fc6-cb3dcff0cf1d service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:29.699342 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e165812-fdac-494a-9fc6-cb3dcff0cf1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:29.699618 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e165812-fdac-494a-9fc6-cb3dcff0cf1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:29.708349 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e165812-fdac-494a-9fc6-cb3dcff0cf1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:32:29.709753 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e165812-fdac-494a-9fc6-cb3dcff0cf1d service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:32:29.718891 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 712/1464] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:32:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:32:31.957897 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0fc173-ae0b-41c6-9ac0-ad2824b83da5 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:31.958460 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0fc173-ae0b-41c6-9ac0-ad2824b83da5 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:31.958460 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0fc173-ae0b-41c6-9ac0-ad2824b83da5 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:31.959865 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 753/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 31 14:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:32:31.965181 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120695f-4a5a-44a8-be65-916c7c56ffc4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:31.965629 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120695f-4a5a-44a8-be65-916c7c56ffc4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:31.965743 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120695f-4a5a-44a8-be65-916c7c56ffc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:31.983535 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1120695f-4a5a-44a8-be65-916c7c56ffc4 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:32:31.984722 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1120695f-4a5a-44a8-be65-916c7c56ffc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:32:31.992475 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 713/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 31 14:32:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:32:32.028016 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0fc173-ae0b-41c6-9ac0-ad2824b83da5 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:32:32.039166 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0fc173-ae0b-41c6-9ac0-ad2824b83da5 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:32:32.042635 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e57c4e-af66-473a-8741-0ff7e442197b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:32:32.043304 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e57c4e-af66-473a-8741-0ff7e442197b service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:32.043588 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e57c4e-af66-473a-8741-0ff7e442197b service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:32.043865 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e57c4e-af66-473a-8741-0ff7e442197b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:32.052569 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e57c4e-af66-473a-8741-0ff7e442197b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:32:32.053997 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e57c4e-af66-473a-8741-0ff7e442197b service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:32:32.063221 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 754/1467] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:32:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:32:34.414328 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecec76-56f0-4bae-974d-4b870d9fe0a4 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:34.414971 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecec76-56f0-4bae-974d-4b870d9fe0a4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:34.414971 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecec76-56f0-4bae-974d-4b870d9fe0a4 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:34.416236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 714/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 31 14:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:32:34.422719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbedf71-a2ec-4bf0-af55-2d91a90fc29c None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:34.423240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbedf71-a2ec-4bf0-af55-2d91a90fc29c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:34.423356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbedf71-a2ec-4bf0-af55-2d91a90fc29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:34.442426 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-ecbedf71-a2ec-4bf0-af55-2d91a90fc29c None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:32:34.443709 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ecbedf71-a2ec-4bf0-af55-2d91a90fc29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:32:34.452022 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 755/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 31 14:32:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:32:34.489646 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecec76-56f0-4bae-974d-4b870d9fe0a4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:32:34.500338 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecec76-56f0-4bae-974d-4b870d9fe0a4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:32:34.503827 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b387b-546d-4491-bab3-6a902b5fdded service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:32:34.504478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b387b-546d-4491-bab3-6a902b5fdded service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:32:34.504764 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b387b-546d-4491-bab3-6a902b5fdded service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:32:34.505043 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b387b-546d-4491-bab3-6a902b5fdded service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:32:34.514057 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b387b-546d-4491-bab3-6a902b5fdded service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:32:34.515520 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b387b-546d-4491-bab3-6a902b5fdded service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:32:34.524975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 715/1470] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:32:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:32:44.436170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=17 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:32:44.520740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=17 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:33:06.490326 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54adf27c-f3b5-4401-beca-370955f0fa23 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:33:06.490984 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54adf27c-f3b5-4401-beca-370955f0fa23 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:33:06.490984 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54adf27c-f3b5-4401-beca-370955f0fa23 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:33:06.492205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 756/1471] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:33:06.497506 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5b6cd-2fd0-42ae-a6b7-efc4fea60bf6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:33:06.498538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5b6cd-2fd0-42ae-a6b7-efc4fea60bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:33:06.498637 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5b6cd-2fd0-42ae-a6b7-efc4fea60bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:33:06.516741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-67d5b6cd-2fd0-42ae-a6b7-efc4fea60bf6 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:33:06.517942 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-67d5b6cd-2fd0-42ae-a6b7-efc4fea60bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:33:06.525981 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 716/1472] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:33:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:33:06.572757 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4ad68-2a14-472e-a48b-f1b77528fb05 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:33:06.573046 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4ad68-2a14-472e-a48b-f1b77528fb05 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:33:06.573323 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4ad68-2a14-472e-a48b-f1b77528fb05 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:33:06.574708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 757/1473] 10.222.0.67 () {54 vars in 962 bytes} [Tue May 31 14:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:33:06.579887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9964948-7127-4a75-8ca9-353098f87fff None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:33:06.580181 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9964948-7127-4a75-8ca9-353098f87fff None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:33:06.580461 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9964948-7127-4a75-8ca9-353098f87fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:33:06.598835 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e9964948-7127-4a75-8ca9-353098f87fff None None] MFA Rules not processed for user `d8a4b0dd4e21414d91fe7bed478d15e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:33:06.600033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e9964948-7127-4a75-8ca9-353098f87fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:33:06.606628 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 717/1474] 10.222.0.67 () {58 vars in 1059 bytes} [Tue May 31 14:33:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:33:06.613752 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4ad68-2a14-472e-a48b-f1b77528fb05 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:33:06.624601 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4ad68-2a14-472e-a48b-f1b77528fb05 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:33:06.628145 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd821a-691b-4d23-a47b-cf2d1cff6b3b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8a4b0dd4e21414d91fe7bed478d15e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:33:06.628785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd821a-691b-4d23-a47b-cf2d1cff6b3b service glance] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:33:06.629064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd821a-691b-4d23-a47b-cf2d1cff6b3b service glance] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:33:06.629334 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd821a-691b-4d23-a47b-cf2d1cff6b3b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:33:06.638006 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd821a-691b-4d23-a47b-cf2d1cff6b3b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:33:06.640010 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd821a-691b-4d23-a47b-cf2d1cff6b3b service glance] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:33:06.649225 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 758/1475] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:33:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:33:09.024981 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9734e97-67cd-4949-bf7f-d60484e186f3 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:33:09.025582 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9734e97-67cd-4949-bf7f-d60484e186f3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:33:09.025582 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9734e97-67cd-4949-bf7f-d60484e186f3 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:33:09.026959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 718/1476] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:33:09.032647 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837e5fb-1030-4b3d-b2c5-79b77239609b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:33:09.033101 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837e5fb-1030-4b3d-b2c5-79b77239609b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:33:09.033247 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837e5fb-1030-4b3d-b2c5-79b77239609b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:33:09.052161 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6837e5fb-1030-4b3d-b2c5-79b77239609b None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:33:09.053357 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6837e5fb-1030-4b3d-b2c5-79b77239609b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:33:09.061971 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 759/1477] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:33:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:33:09.088647 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9734e97-67cd-4949-bf7f-d60484e186f3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:33:09.100998 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9734e97-67cd-4949-bf7f-d60484e186f3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:33:09.105391 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6a395-7525-4fec-b5fb-546d1f6e32f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:33:09.106035 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6a395-7525-4fec-b5fb-546d1f6e32f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:33:09.106319 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6a395-7525-4fec-b5fb-546d1f6e32f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:33:09.106598 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6a395-7525-4fec-b5fb-546d1f6e32f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:33:09.107383 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6a395-7525-4fec-b5fb-546d1f6e32f0 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:33:09.109136 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-dae6a395-7525-4fec-b5fb-546d1f6e32f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 31 14:33:09.109136 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: warnings.warn(msg) May 31 14:33:09.109136 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: May 31 14:33:09.110249 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6a395-7525-4fec-b5fb-546d1f6e32f0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:33:09.130483 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 719/1478] 10.222.0.67 () {56 vars in 1108 bytes} [Tue May 31 14:33:09 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:33:11.395342 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e549b67-5b0b-4f52-8f55-53f0b3832ec9 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:33:11.395904 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e549b67-5b0b-4f52-8f55-53f0b3832ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:33:11.395904 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e549b67-5b0b-4f52-8f55-53f0b3832ec9 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:33:11.397281 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 760/1479] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:33:11.402433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627b293-bb9f-4c56-bebd-dac9c9d6fc96 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:33:11.402857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627b293-bb9f-4c56-bebd-dac9c9d6fc96 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:33:11.403031 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627b293-bb9f-4c56-bebd-dac9c9d6fc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:33:11.420643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f627b293-bb9f-4c56-bebd-dac9c9d6fc96 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:33:11.421836 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f627b293-bb9f-4c56-bebd-dac9c9d6fc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:33:11.429538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 720/1480] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:33:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:33:11.473883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e549b67-5b0b-4f52-8f55-53f0b3832ec9 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:33:11.488681 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e549b67-5b0b-4f52-8f55-53f0b3832ec9 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:33:11.494121 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0a340-0f31-4aa5-8710-13a1a0d67c1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:33:11.494790 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0a340-0f31-4aa5-8710-13a1a0d67c1c service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:33:11.495097 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0a340-0f31-4aa5-8710-13a1a0d67c1c service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:33:11.495392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0a340-0f31-4aa5-8710-13a1a0d67c1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:33:11.504142 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce0a340-0f31-4aa5-8710-13a1a0d67c1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:33:11.505823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce0a340-0f31-4aa5-8710-13a1a0d67c1c service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:33:11.515618 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 761/1481] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:33:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:33:14.026432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7acf4a-4b50-41cb-bd1c-d3712ca322e1 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:33:14.027005 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7acf4a-4b50-41cb-bd1c-d3712ca322e1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:33:14.027005 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7acf4a-4b50-41cb-bd1c-d3712ca322e1 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:33:14.028554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 721/1482] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:33:14.034560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c7657-dbe8-44c4-92ab-801768ec5234 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:33:14.035040 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c7657-dbe8-44c4-92ab-801768ec5234 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:33:14.035194 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c7657-dbe8-44c4-92ab-801768ec5234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:33:14.054603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-022c7657-dbe8-44c4-92ab-801768ec5234 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:33:14.056000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-022c7657-dbe8-44c4-92ab-801768ec5234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:33:14.064819 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 762/1483] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:33:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:33:14.104082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7acf4a-4b50-41cb-bd1c-d3712ca322e1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:33:14.116337 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7acf4a-4b50-41cb-bd1c-d3712ca322e1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:33:14.120159 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b85fce-b802-49f5-950d-59edcab6ed7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:33:14.120818 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b85fce-b802-49f5-950d-59edcab6ed7d service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:33:14.121102 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b85fce-b802-49f5-950d-59edcab6ed7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:33:14.121379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b85fce-b802-49f5-950d-59edcab6ed7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:33:14.133273 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b85fce-b802-49f5-950d-59edcab6ed7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:33:14.134814 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b85fce-b802-49f5-950d-59edcab6ed7d service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:33:14.145311 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 722/1484] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:33:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:33:16.444492 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4eb5a-dd6f-4ce1-86f9-b9967c9c6273 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:33:16.445286 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4eb5a-dd6f-4ce1-86f9-b9967c9c6273 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:33:16.445286 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4eb5a-dd6f-4ce1-86f9-b9967c9c6273 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:33:16.446636 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 763/1485] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:33:16.452252 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7064e-7de3-45a1-ac5a-d6bc37785ba5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:33:16.452715 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7064e-7de3-45a1-ac5a-d6bc37785ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:33:16.452823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7064e-7de3-45a1-ac5a-d6bc37785ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:33:16.471269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-96b7064e-7de3-45a1-ac5a-d6bc37785ba5 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:33:16.472489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-96b7064e-7de3-45a1-ac5a-d6bc37785ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:33:16.480435 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 723/1486] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:33:16.518453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4eb5a-dd6f-4ce1-86f9-b9967c9c6273 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:33:16.530094 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4eb5a-dd6f-4ce1-86f9-b9967c9c6273 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:33:16.533755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6b2d7-6c0d-42b0-9af8-b123482515b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:33:16.534398 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6b2d7-6c0d-42b0-9af8-b123482515b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:33:16.534700 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6b2d7-6c0d-42b0-9af8-b123482515b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:33:16.534977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6b2d7-6c0d-42b0-9af8-b123482515b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:33:16.545573 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6b2d7-6c0d-42b0-9af8-b123482515b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:33:16.547053 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6b2d7-6c0d-42b0-9af8-b123482515b0 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:33:16.557321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 764/1487] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:33:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:33:18.999562 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf82191-b2ed-4a21-9600-a79c03032b7e None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:33:19.000078 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf82191-b2ed-4a21-9600-a79c03032b7e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:33:19.000078 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf82191-b2ed-4a21-9600-a79c03032b7e None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:33:19.001595 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 724/1488] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:33:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:33:19.007419 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc927e5-3b2d-46a4-b576-01357dac468b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:33:19.008038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc927e5-3b2d-46a4-b576-01357dac468b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:33:19.008486 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc927e5-3b2d-46a4-b576-01357dac468b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:33:19.034258 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-9fc927e5-3b2d-46a4-b576-01357dac468b None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:33:19.036140 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9fc927e5-3b2d-46a4-b576-01357dac468b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:33:19.046856 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 765/1489] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:33:19.092024 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf82191-b2ed-4a21-9600-a79c03032b7e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:33:19.103437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf82191-b2ed-4a21-9600-a79c03032b7e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:33:19.107723 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a5331-92e8-4606-baba-ad32b549002c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:33:19.108384 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a5331-92e8-4606-baba-ad32b549002c service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:33:19.108661 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a5331-92e8-4606-baba-ad32b549002c service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:33:19.108943 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a5331-92e8-4606-baba-ad32b549002c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:33:19.118610 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a5331-92e8-4606-baba-ad32b549002c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:33:19.120080 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a5331-92e8-4606-baba-ad32b549002c service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:33:19.130722 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 725/1490] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:33:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:33:21.787207 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5e9d6-1349-4da4-83ea-f7c652435855 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:33:21.787808 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5e9d6-1349-4da4-83ea-f7c652435855 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:33:21.787808 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5e9d6-1349-4da4-83ea-f7c652435855 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:33:21.789160 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 766/1491] 10.222.0.67 () {54 vars in 926 bytes} [Tue May 31 14:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:33:21.794388 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc127c9-220a-4cfa-ad30-25693a22d06d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:33:21.794834 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc127c9-220a-4cfa-ad30-25693a22d06d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:33:21.795002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc127c9-220a-4cfa-ad30-25693a22d06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:33:21.813032 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1cc127c9-220a-4cfa-ad30-25693a22d06d None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:33:21.814241 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1cc127c9-220a-4cfa-ad30-25693a22d06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:33:21.822484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 726/1492] 10.222.0.67 () {58 vars in 1023 bytes} [Tue May 31 14:33:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:33:21.861559 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e5e9d6-1349-4da4-83ea-f7c652435855 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:33:21.873163 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e5e9d6-1349-4da4-83ea-f7c652435855 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:33:21.876726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008184c-ca1f-4da5-90b5-196d63acee17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:33:21.877359 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008184c-ca1f-4da5-90b5-196d63acee17 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:33:21.877642 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008184c-ca1f-4da5-90b5-196d63acee17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:33:21.877918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008184c-ca1f-4da5-90b5-196d63acee17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:33:21.886810 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008184c-ca1f-4da5-90b5-196d63acee17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:33:21.888866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008184c-ca1f-4da5-90b5-196d63acee17 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:33:21.898120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 767/1493] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:33:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:33:31.807348 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=33 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:33:31.894029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=33 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:34:54.319511 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3e7e6-0b59-4ada-b6a1-66a207bc9d2e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:34:54.320109 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3e7e6-0b59-4ada-b6a1-66a207bc9d2e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:34:54.320109 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3e7e6-0b59-4ada-b6a1-66a207bc9d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:34:54.338854 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b6d3e7e6-0b59-4ada-b6a1-66a207bc9d2e None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:34:54.340054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b6d3e7e6-0b59-4ada-b6a1-66a207bc9d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:34:54.347867 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 727/1494] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:34:54.356407 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee038c40-c693-4989-b59b-7a7ba297e200 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:34:54.356957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee038c40-c693-4989-b59b-7a7ba297e200 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:34:54.356957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee038c40-c693-4989-b59b-7a7ba297e200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:34:54.375545 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-ee038c40-c693-4989-b59b-7a7ba297e200 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:34:54.377320 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ee038c40-c693-4989-b59b-7a7ba297e200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:34:54.387639 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 768/1495] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:34:54.392344 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3e7e6-0b59-4ada-b6a1-66a207bc9d2e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:34:54.406975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3e7e6-0b59-4ada-b6a1-66a207bc9d2e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:34:54.413238 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a89692-17c5-4558-a39b-4a856fe0676c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:34:54.413905 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a89692-17c5-4558-a39b-4a856fe0676c admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:34:54.414223 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a89692-17c5-4558-a39b-4a856fe0676c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:34:54.414502 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a89692-17c5-4558-a39b-4a856fe0676c admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:34:54.415266 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a89692-17c5-4558-a39b-4a856fe0676c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:34:54.415579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a89692-17c5-4558-a39b-4a856fe0676c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:34:54.416602 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-79a89692-17c5-4558-a39b-4a856fe0676c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 31 14:34:54.416602 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: warnings.warn(msg) May 31 14:34:54.416602 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: May 31 14:34:54.417976 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a89692-17c5-4558-a39b-4a856fe0676c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:34:54.429639 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 728/1496] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:34:54.438413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee038c40-c693-4989-b59b-7a7ba297e200 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:34:54.448233 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee038c40-c693-4989-b59b-7a7ba297e200 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:34:54.452753 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480dfed5-ddf0-4eee-938b-507787bd425c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:34:54.453404 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480dfed5-ddf0-4eee-938b-507787bd425c admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:34:54.453710 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480dfed5-ddf0-4eee-938b-507787bd425c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:34:54.454008 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480dfed5-ddf0-4eee-938b-507787bd425c admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:34:54.454732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480dfed5-ddf0-4eee-938b-507787bd425c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:34:54.456495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: WARNING py.warnings [None req-480dfed5-ddf0-4eee-938b-507787bd425c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 31 14:34:54.456495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: warnings.warn(msg) May 31 14:34:54.456495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: May 31 14:34:54.457760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480dfed5-ddf0-4eee-938b-507787bd425c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:34:54.493817 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 769/1497] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:34:54 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:34:54.498588 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a89692-17c5-4558-a39b-4a856fe0676c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:34:54.508488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a89692-17c5-4558-a39b-4a856fe0676c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:34:54.563587 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc8f13-2569-4c5b-a0c8-7c02fbfd4e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:34:54.564258 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc8f13-2569-4c5b-a0c8-7c02fbfd4e55 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:34:54.564547 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc8f13-2569-4c5b-a0c8-7c02fbfd4e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:34:54.564831 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc8f13-2569-4c5b-a0c8-7c02fbfd4e55 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:34:54.565528 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc8f13-2569-4c5b-a0c8-7c02fbfd4e55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:34:54.566550 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-82cc8f13-2569-4c5b-a0c8-7c02fbfd4e55 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 31 14:34:54.566550 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: warnings.warn(msg) May 31 14:34:54.566550 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: May 31 14:34:54.567784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc8f13-2569-4c5b-a0c8-7c02fbfd4e55 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:34:54.587264 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 729/1498] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:34:54 2022] POST /identity/v3/users => generated 289 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:34:54.591844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480dfed5-ddf0-4eee-938b-507787bd425c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:34:54.601764 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480dfed5-ddf0-4eee-938b-507787bd425c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:34:54.606753 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa083e-a599-4d61-b2bb-575ff4909263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:34:54.607404 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa083e-a599-4d61-b2bb-575ff4909263 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:34:54.607685 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa083e-a599-4d61-b2bb-575ff4909263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:34:54.607959 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa083e-a599-4d61-b2bb-575ff4909263 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:34:54.608708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa083e-a599-4d61-b2bb-575ff4909263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:34:54.609830 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: WARNING py.warnings [None req-f2aa083e-a599-4d61-b2bb-575ff4909263 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 31 14:34:54.609830 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: warnings.warn(msg) May 31 14:34:54.609830 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: May 31 14:34:54.610783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa083e-a599-4d61-b2bb-575ff4909263 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:34:54.623266 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 770/1499] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:34:54.627019 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc8f13-2569-4c5b-a0c8-7c02fbfd4e55 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:34:54.636806 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc8f13-2569-4c5b-a0c8-7c02fbfd4e55 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:34:54.641434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072be2be-b8c7-4246-be61-4b6d7ff80f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:34:54.642030 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072be2be-b8c7-4246-be61-4b6d7ff80f07 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:34:54.642319 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072be2be-b8c7-4246-be61-4b6d7ff80f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:34:54.642605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072be2be-b8c7-4246-be61-4b6d7ff80f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:34:54.643368 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072be2be-b8c7-4246-be61-4b6d7ff80f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:34:54.644374 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-072be2be-b8c7-4246-be61-4b6d7ff80f07 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 31 14:34:54.644374 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: warnings.warn(msg) May 31 14:34:54.644374 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: May 31 14:34:54.645250 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072be2be-b8c7-4246-be61-4b6d7ff80f07 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:34:54.658898 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 730/1500] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:34:54.662756 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa083e-a599-4d61-b2bb-575ff4909263 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:34:54.673043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa083e-a599-4d61-b2bb-575ff4909263 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:34:54.677809 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca85df1-c8db-4adb-8fb5-c3730c089a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:34:54.678752 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca85df1-c8db-4adb-8fb5-c3730c089a4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:34:54.679036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca85df1-c8db-4adb-8fb5-c3730c089a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:34:54.679310 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca85df1-c8db-4adb-8fb5-c3730c089a4e admin admin] PATH_INFO: `/v3/projects/ce9f9b771b2b48fd8970018ad83db5cf/users/28d3cf2bf1cb45fc8acb2c95c1283fed/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:34:54.693719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca85df1-c8db-4adb-8fb5-c3730c089a4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce9f9b771b2b48fd8970018ad83db5cf, user_id=28d3cf2bf1cb45fc8acb2c95c1283fed, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:34:54.695034 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: WARNING py.warnings [None req-bca85df1-c8db-4adb-8fb5-c3730c089a4e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 31 14:34:54.695034 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: warnings.warn(msg) May 31 14:34:54.695034 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: May 31 14:34:54.696923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca85df1-c8db-4adb-8fb5-c3730c089a4e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:34:54.714185 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 771/1501] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:34:54 2022] PUT /identity/v3/projects/ce9f9b771b2b48fd8970018ad83db5cf/users/28d3cf2bf1cb45fc8acb2c95c1283fed/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:34:54.726142 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee06f58-fe4b-40df-84b9-4a5a8b07d9a2 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:34:54.726439 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee06f58-fe4b-40df-84b9-4a5a8b07d9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:34:54.726780 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee06f58-fe4b-40df-84b9-4a5a8b07d9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:34:54.758038 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6ee06f58-fe4b-40df-84b9-4a5a8b07d9a2 None None] MFA Rules not processed for user `28d3cf2bf1cb45fc8acb2c95c1283fed`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:34:54.759639 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6ee06f58-fe4b-40df-84b9-4a5a8b07d9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:34:54.823528 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 731/1502] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:34:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:34:54.836774 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51091ea2-2de7-49b4-bfdf-4e10c45e3ae5 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:34:54.837060 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51091ea2-2de7-49b4-bfdf-4e10c45e3ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:34:54.837335 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51091ea2-2de7-49b4-bfdf-4e10c45e3ae5 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:34:54.839236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 772/1503] 10.222.0.67 () {54 vars in 963 bytes} [Tue May 31 14:34:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:34:54.844193 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14d11f-c543-400b-b3de-47b8c264dc87 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:34:54.844488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14d11f-c543-400b-b3de-47b8c264dc87 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:34:54.844799 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14d11f-c543-400b-b3de-47b8c264dc87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:34:54.863244 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ef14d11f-c543-400b-b3de-47b8c264dc87 None None] MFA Rules not processed for user `990999672d1a4f19896a143e1eeddd44`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:34:54.864434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ef14d11f-c543-400b-b3de-47b8c264dc87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:34:54.906585 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:34:54.915091 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 732/1504] 10.222.0.67 () {58 vars in 1060 bytes} [Tue May 31 14:34:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:34:54.922924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51091ea2-2de7-49b4-bfdf-4e10c45e3ae5 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:34:54.933171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51091ea2-2de7-49b4-bfdf-4e10c45e3ae5 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:34:54.936606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57711779-78bc-42c3-b91c-8685855a2f3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:34:54.937244 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57711779-78bc-42c3-b91c-8685855a2f3b service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:34:54.937508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57711779-78bc-42c3-b91c-8685855a2f3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:34:54.937781 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57711779-78bc-42c3-b91c-8685855a2f3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:34:54.946413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57711779-78bc-42c3-b91c-8685855a2f3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:34:54.947895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57711779-78bc-42c3-b91c-8685855a2f3b service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:34:54.958143 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 773/1505] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:34:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:34:55.001550 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b958007-8075-48a5-9cc9-1968aa0a5a44 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:34:55.001839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b958007-8075-48a5-9cc9-1968aa0a5a44 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:34:55.002123 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b958007-8075-48a5-9cc9-1968aa0a5a44 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:34:55.003443 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 733/1506] 10.222.0.67 () {52 vars in 829 bytes} [Tue May 31 14:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:34:55.007109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57711779-78bc-42c3-b91c-8685855a2f3b service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:34:55.067669 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57711779-78bc-42c3-b91c-8685855a2f3b service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:34:55.072327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3d8a1-cd68-4a08-b869-a73f43859b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:34:55.073681 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3d8a1-cd68-4a08-b869-a73f43859b44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:34:55.073962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3d8a1-cd68-4a08-b869-a73f43859b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:34:55.074235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3d8a1-cd68-4a08-b869-a73f43859b44 admin admin] PATH_INFO: `/v3/users/28d3cf2bf1cb45fc8acb2c95c1283fed` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:34:55.075441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3d8a1-cd68-4a08-b869-a73f43859b44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28d3cf2bf1cb45fc8acb2c95c1283fed)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:34:55.076457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: WARNING py.warnings [None req-6bc3d8a1-cd68-4a08-b869-a73f43859b44 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 31 14:34:55.076457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: warnings.warn(msg) May 31 14:34:55.076457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: May 31 14:34:55.077631 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3d8a1-cd68-4a08-b869-a73f43859b44 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:34:55.141645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6bc3d8a1-cd68-4a08-b869-a73f43859b44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28d3cf2bf1cb45fc8acb2c95c1283fed'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:34:55.150702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6bc3d8a1-cd68-4a08-b869-a73f43859b44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28d3cf2bf1cb45fc8acb2c95c1283fed'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:34:55.151952 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6bc3d8a1-cd68-4a08-b869-a73f43859b44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28d3cf2bf1cb45fc8acb2c95c1283fed'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:34:55.169691 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6bc3d8a1-cd68-4a08-b869-a73f43859b44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28d3cf2bf1cb45fc8acb2c95c1283fed'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:34:55.170398 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6bc3d8a1-cd68-4a08-b869-a73f43859b44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28d3cf2bf1cb45fc8acb2c95c1283fed'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:34:55.180189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:34:55.191557 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 774/1507] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:34:55 2022] DELETE /identity/v3/users/28d3cf2bf1cb45fc8acb2c95c1283fed => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:34:55.196179 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2efe5f-ff28-4bd6-82ad-e870eea2aaa7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:34:55.196466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2efe5f-ff28-4bd6-82ad-e870eea2aaa7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:34:55.196740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2efe5f-ff28-4bd6-82ad-e870eea2aaa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:34:55.215087 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9a2efe5f-ff28-4bd6-82ad-e870eea2aaa7 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:34:55.216296 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9a2efe5f-ff28-4bd6-82ad-e870eea2aaa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:34:55.272896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 734/1508] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:34:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:34:55.282885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3d8a1-cd68-4a08-b869-a73f43859b44 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:34:55.284272 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6bc3d8a1-cd68-4a08-b869-a73f43859b44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:34:55.340996 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3d8a1-cd68-4a08-b869-a73f43859b44 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:34:55.344571 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686198d-bb40-43ce-8c7e-689a773f578c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:34:55.345214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686198d-bb40-43ce-8c7e-689a773f578c service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:34:55.345502 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686198d-bb40-43ce-8c7e-689a773f578c service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:34:55.345776 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686198d-bb40-43ce-8c7e-689a773f578c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:34:55.354510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686198d-bb40-43ce-8c7e-689a773f578c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:34:55.355977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686198d-bb40-43ce-8c7e-689a773f578c service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:34:55.365230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 775/1509] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:34:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:34:55.711504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2efe5f-ff28-4bd6-82ad-e870eea2aaa7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:34:55.713022 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9a2efe5f-ff28-4bd6-82ad-e870eea2aaa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:34:55.725377 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2efe5f-ff28-4bd6-82ad-e870eea2aaa7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:34:55.731006 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6f7fe-f7bb-4c13-ad93-19ab21634519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:34:55.732019 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6f7fe-f7bb-4c13-ad93-19ab21634519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:34:55.732303 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6f7fe-f7bb-4c13-ad93-19ab21634519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:34:55.732579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6f7fe-f7bb-4c13-ad93-19ab21634519 admin admin] PATH_INFO: `/v3/projects/ce9f9b771b2b48fd8970018ad83db5cf` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:34:55.733783 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6f7fe-f7bb-4c13-ad93-19ab21634519 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce9f9b771b2b48fd8970018ad83db5cf)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:34:55.735513 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-e5c6f7fe-f7bb-4c13-ad93-19ab21634519 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 31 14:34:55.735513 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: warnings.warn(msg) May 31 14:34:55.735513 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: May 31 14:34:55.736832 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6f7fe-f7bb-4c13-ad93-19ab21634519 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:34:55.797116 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e5c6f7fe-f7bb-4c13-ad93-19ab21634519 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce9f9b771b2b48fd8970018ad83db5cf', 'request_id': 'req-e5c6f7fe-f7bb-4c13-ad93-19ab21634519'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:34:55.802952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e5c6f7fe-f7bb-4c13-ad93-19ab21634519 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce9f9b771b2b48fd8970018ad83db5cf', 'request_id': 'req-e5c6f7fe-f7bb-4c13-ad93-19ab21634519'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:34:55.818975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e5c6f7fe-f7bb-4c13-ad93-19ab21634519 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce9f9b771b2b48fd8970018ad83db5cf', 'request_id': 'req-e5c6f7fe-f7bb-4c13-ad93-19ab21634519'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:34:55.834126 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e5c6f7fe-f7bb-4c13-ad93-19ab21634519 admin admin] The token cache is being invalidate because project ce9f9b771b2b48fd8970018ad83db5cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:34:55.834419 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e5c6f7fe-f7bb-4c13-ad93-19ab21634519 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:34:55.835089 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e5c6f7fe-f7bb-4c13-ad93-19ab21634519 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:34:55.836770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 735/1510] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:34:55 2022] DELETE /identity/v3/projects/ce9f9b771b2b48fd8970018ad83db5cf => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:35:05.361138 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=29 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:35:05.831154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:36:31.981158 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9543b24-80b2-4b11-937a-3801c3b1e514 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:31.981754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9543b24-80b2-4b11-937a-3801c3b1e514 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:31.981754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9543b24-80b2-4b11-937a-3801c3b1e514 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:31.983441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 776/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 31 14:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:36:31.988892 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb8fbb-c30d-4713-a9a0-ada01a5e39c1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:31.989381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb8fbb-c30d-4713-a9a0-ada01a5e39c1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:31.989491 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb8fbb-c30d-4713-a9a0-ada01a5e39c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:32.022958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-16bb8fbb-c30d-4713-a9a0-ada01a5e39c1 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:36:32.024176 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-16bb8fbb-c30d-4713-a9a0-ada01a5e39c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:36:32.081687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 736/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 31 14:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:36:32.247352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9543b24-80b2-4b11-937a-3801c3b1e514 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:36:32.249031 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c9543b24-80b2-4b11-937a-3801c3b1e514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:36:32.330069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9543b24-80b2-4b11-937a-3801c3b1e514 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:36:32.336845 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68300508-1173-4803-8642-851210f99577 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:36:32.337711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68300508-1173-4803-8642-851210f99577 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:32.337841 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68300508-1173-4803-8642-851210f99577 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:32.338061 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68300508-1173-4803-8642-851210f99577 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:32.351292 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68300508-1173-4803-8642-851210f99577 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:36:32.352900 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68300508-1173-4803-8642-851210f99577 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:36:32.363907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 777/1513] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:36:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:36:33.233216 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46325045-50ec-4cad-a41e-3a6e611559b6 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:33.233757 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46325045-50ec-4cad-a41e-3a6e611559b6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:33.233757 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46325045-50ec-4cad-a41e-3a6e611559b6 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:33.235240 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 737/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 31 14:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:36:33.240305 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106ff25-b2b9-488a-b0a0-cfc4853effee None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:33.240712 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106ff25-b2b9-488a-b0a0-cfc4853effee None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:33.240868 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106ff25-b2b9-488a-b0a0-cfc4853effee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:33.259197 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5106ff25-b2b9-488a-b0a0-cfc4853effee None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:36:33.260404 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5106ff25-b2b9-488a-b0a0-cfc4853effee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:36:33.269080 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 778/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 31 14:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:36:33.418277 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46325045-50ec-4cad-a41e-3a6e611559b6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:36:33.433062 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46325045-50ec-4cad-a41e-3a6e611559b6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:36:33.438991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d8d4f4-15da-4625-90f9-c7a37f6977a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:36:33.439637 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8d4f4-15da-4625-90f9-c7a37f6977a5 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:33.440568 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8d4f4-15da-4625-90f9-c7a37f6977a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:33.440827 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8d4f4-15da-4625-90f9-c7a37f6977a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:33.449740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d8d4f4-15da-4625-90f9-c7a37f6977a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:36:33.451440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d8d4f4-15da-4625-90f9-c7a37f6977a5 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:36:33.461291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 738/1516] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:36:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:36:34.172666 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bc05c-3126-436f-bcd6-d3ee03212849 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:34.172953 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bc05c-3126-436f-bcd6-d3ee03212849 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:34.173230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bc05c-3126-436f-bcd6-d3ee03212849 None None] PATH_INFO: `/v3` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:34.175279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 779/1517] 10.222.0.79 () {54 vars in 925 bytes} [Tue May 31 14:36:34 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:36:34.181560 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3791d6c-efc0-4629-9b9d-157ed39ce6a8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:34.181857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3791d6c-efc0-4629-9b9d-157ed39ce6a8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:34.182143 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3791d6c-efc0-4629-9b9d-157ed39ce6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:34.200269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f3791d6c-efc0-4629-9b9d-157ed39ce6a8 None None] MFA Rules not processed for user `6c1c13515a8b41a781478dc65e6a88b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:36:34.201533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f3791d6c-efc0-4629-9b9d-157ed39ce6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:36:34.264977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 739/1518] 10.222.0.79 () {58 vars in 1015 bytes} [Tue May 31 14:36:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:36:38.278375 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0d470-0908-4741-8c2e-2eec4848c5a9 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:38.278965 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0d470-0908-4741-8c2e-2eec4848c5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:38.278965 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0d470-0908-4741-8c2e-2eec4848c5a9 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:38.280354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 780/1519] 10.222.0.67 () {54 vars in 960 bytes} [Tue May 31 14:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:36:38.286340 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a82dcb-6e08-4e6b-97d4-5bfe19f46c38 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:38.286804 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a82dcb-6e08-4e6b-97d4-5bfe19f46c38 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:38.286941 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a82dcb-6e08-4e6b-97d4-5bfe19f46c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:38.305091 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-50a82dcb-6e08-4e6b-97d4-5bfe19f46c38 None None] MFA Rules not processed for user `6c1c13515a8b41a781478dc65e6a88b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:36:38.306295 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-50a82dcb-6e08-4e6b-97d4-5bfe19f46c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:36:38.314752 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 740/1520] 10.222.0.67 () {58 vars in 1057 bytes} [Tue May 31 14:36:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:36:38.322879 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0d470-0908-4741-8c2e-2eec4848c5a9 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:36:38.336629 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0d470-0908-4741-8c2e-2eec4848c5a9 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:36:38.341496 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d41cb-e3e6-4103-9248-419163697ad7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c1c13515a8b41a781478dc65e6a88b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:36:38.342164 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d41cb-e3e6-4103-9248-419163697ad7 service placement] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:38.342457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d41cb-e3e6-4103-9248-419163697ad7 service placement] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:38.342781 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d41cb-e3e6-4103-9248-419163697ad7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:38.351155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d41cb-e3e6-4103-9248-419163697ad7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:36:38.352748 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d41cb-e3e6-4103-9248-419163697ad7 service placement] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:36:38.362328 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 781/1521] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:36:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:36:44.750177 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd095a-cde8-485f-b3df-3f768ba9c2aa None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:44.750877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd095a-cde8-485f-b3df-3f768ba9c2aa None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:44.750877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd095a-cde8-485f-b3df-3f768ba9c2aa None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:44.752686 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 741/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 31 14:36:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:36:44.757392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190e8f6-5a6f-4558-bb49-f19e1ca236af None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:44.757833 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190e8f6-5a6f-4558-bb49-f19e1ca236af None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:44.757936 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190e8f6-5a6f-4558-bb49-f19e1ca236af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:44.776056 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-f190e8f6-5a6f-4558-bb49-f19e1ca236af None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:36:44.777323 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f190e8f6-5a6f-4558-bb49-f19e1ca236af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:36:44.785618 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 782/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 31 14:36:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:36:44.935236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd095a-cde8-485f-b3df-3f768ba9c2aa None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:36:44.949997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd095a-cde8-485f-b3df-3f768ba9c2aa None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:36:44.955671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2f496-13ac-4061-a863-4eeeac0ee1bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:36:44.956805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2f496-13ac-4061-a863-4eeeac0ee1bc service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:44.957092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2f496-13ac-4061-a863-4eeeac0ee1bc service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:44.957374 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2f496-13ac-4061-a863-4eeeac0ee1bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:44.966106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2f496-13ac-4061-a863-4eeeac0ee1bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:36:44.967803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2f496-13ac-4061-a863-4eeeac0ee1bc service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:36:44.978012 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 742/1524] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:36:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:36:45.836579 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa47ab-b76a-49e2-b6f2-10649fa3cd96 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:45.837015 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa47ab-b76a-49e2-b6f2-10649fa3cd96 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:45.837146 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa47ab-b76a-49e2-b6f2-10649fa3cd96 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:45.838547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 783/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 31 14:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:36:45.843478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b3088-7b33-4730-972e-e7ee0be3d007 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:45.843949 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b3088-7b33-4730-972e-e7ee0be3d007 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:45.844061 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b3088-7b33-4730-972e-e7ee0be3d007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:45.863586 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-055b3088-7b33-4730-972e-e7ee0be3d007 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:36:45.864798 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-055b3088-7b33-4730-972e-e7ee0be3d007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:36:45.872598 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 743/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 31 14:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:36:46.028384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa47ab-b76a-49e2-b6f2-10649fa3cd96 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:36:46.046242 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa47ab-b76a-49e2-b6f2-10649fa3cd96 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:36:46.052302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2028ed64-2023-44b7-882b-53fc66300047 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:36:46.052944 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028ed64-2023-44b7-882b-53fc66300047 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:46.053231 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028ed64-2023-44b7-882b-53fc66300047 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:46.053513 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028ed64-2023-44b7-882b-53fc66300047 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:46.061828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2028ed64-2023-44b7-882b-53fc66300047 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:36:46.063441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2028ed64-2023-44b7-882b-53fc66300047 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:36:46.073186 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 784/1527] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:36:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:36:50.307808 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e53d30-f4ff-4f34-a832-920a0fcfd48c None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:50.308373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e53d30-f4ff-4f34-a832-920a0fcfd48c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:50.308373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e53d30-f4ff-4f34-a832-920a0fcfd48c None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:50.309725 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 744/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 31 14:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:36:50.315064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb219c-bc09-403d-9a9c-46977aaddbfa None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:50.315555 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb219c-bc09-403d-9a9c-46977aaddbfa None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:50.315662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb219c-bc09-403d-9a9c-46977aaddbfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:50.333659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-fbeb219c-bc09-403d-9a9c-46977aaddbfa None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:36:50.334893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-fbeb219c-bc09-403d-9a9c-46977aaddbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:36:50.342555 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 785/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 31 14:36:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:36:50.379154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e53d30-f4ff-4f34-a832-920a0fcfd48c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:36:50.380874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-42e53d30-f4ff-4f34-a832-920a0fcfd48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:36:50.436344 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e53d30-f4ff-4f34-a832-920a0fcfd48c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:36:50.439831 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e49e30-5805-4317-95f2-6007972082e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:36:50.440468 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e49e30-5805-4317-95f2-6007972082e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:50.440750 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e49e30-5805-4317-95f2-6007972082e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:50.441027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e49e30-5805-4317-95f2-6007972082e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:50.450176 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e49e30-5805-4317-95f2-6007972082e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:36:50.451617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e49e30-5805-4317-95f2-6007972082e3 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:36:50.456439 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:36:50.461936 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 745/1530] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:36:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:36:52.741404 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3345c-2f92-420a-be04-f0b588c8f78d None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:52.741835 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3345c-2f92-420a-be04-f0b588c8f78d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:52.742010 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3345c-2f92-420a-be04-f0b588c8f78d None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:52.743485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 786/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 31 14:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:36:52.748745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3e514-f6b4-4157-af77-9b5165a4c718 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:52.749153 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3e514-f6b4-4157-af77-9b5165a4c718 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:52.749317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3e514-f6b4-4157-af77-9b5165a4c718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:52.767231 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-33a3e514-f6b4-4157-af77-9b5165a4c718 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:36:52.768557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-33a3e514-f6b4-4157-af77-9b5165a4c718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:36:52.776329 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 746/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 31 14:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:36:52.813200 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3345c-2f92-420a-be04-f0b588c8f78d None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:36:52.814930 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-99b3345c-2f92-420a-be04-f0b588c8f78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:36:52.829600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3345c-2f92-420a-be04-f0b588c8f78d None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:36:52.833178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315b74b-fb58-405a-b1b5-f1071aa81330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:36:52.833807 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315b74b-fb58-405a-b1b5-f1071aa81330 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:52.834090 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315b74b-fb58-405a-b1b5-f1071aa81330 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:52.834361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315b74b-fb58-405a-b1b5-f1071aa81330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:52.843239 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f315b74b-fb58-405a-b1b5-f1071aa81330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:36:52.844655 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f315b74b-fb58-405a-b1b5-f1071aa81330 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:36:52.854388 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 787/1533] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:36:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:36:56.745495 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c351fe-ae04-4cbb-b31e-887021ed5654 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:56.745970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c351fe-ae04-4cbb-b31e-887021ed5654 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:56.746076 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c351fe-ae04-4cbb-b31e-887021ed5654 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:56.747472 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 747/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 31 14:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:36:56.752832 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ca527-51c4-4ddc-9a9b-adcbc3fe89c0 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:56.753264 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ca527-51c4-4ddc-9a9b-adcbc3fe89c0 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:56.753388 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ca527-51c4-4ddc-9a9b-adcbc3fe89c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:56.759908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=62 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:36:56.773196 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5f6ca527-51c4-4ddc-9a9b-adcbc3fe89c0 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:36:56.774413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5f6ca527-51c4-4ddc-9a9b-adcbc3fe89c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:36:56.782553 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 788/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 31 14:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:36:57.058477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c351fe-ae04-4cbb-b31e-887021ed5654 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:36:57.060184 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e8c351fe-ae04-4cbb-b31e-887021ed5654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:36:57.116736 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c351fe-ae04-4cbb-b31e-887021ed5654 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:36:57.120492 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da27ee-7b1c-4476-aaa1-acebfecb49cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:36:57.121149 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da27ee-7b1c-4476-aaa1-acebfecb49cb service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:57.121438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da27ee-7b1c-4476-aaa1-acebfecb49cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:57.121715 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da27ee-7b1c-4476-aaa1-acebfecb49cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:57.131776 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da27ee-7b1c-4476-aaa1-acebfecb49cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:36:57.133201 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da27ee-7b1c-4476-aaa1-acebfecb49cb service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:36:57.148372 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 748/1536] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:36:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:36:59.439559 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd907a-281f-4069-9892-a8c33c864bf2 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:59.440601 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd907a-281f-4069-9892-a8c33c864bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:59.440601 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd907a-281f-4069-9892-a8c33c864bf2 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:59.441900 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 789/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 31 14:36:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:36:59.447416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e6e30-1544-4325-88a7-179bcda82e7f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:59.447830 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e6e30-1544-4325-88a7-179bcda82e7f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:59.447992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e6e30-1544-4325-88a7-179bcda82e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:59.466401 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-532e6e30-1544-4325-88a7-179bcda82e7f None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:36:59.467735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-532e6e30-1544-4325-88a7-179bcda82e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:36:59.475539 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 749/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 31 14:36:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:36:59.741885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fd907a-281f-4069-9892-a8c33c864bf2 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:36:59.743635 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-32fd907a-281f-4069-9892-a8c33c864bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:36:59.759994 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fd907a-281f-4069-9892-a8c33c864bf2 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:36:59.763585 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644dff18-af48-4abb-bbf6-a5edc14da680 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:36:59.764226 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644dff18-af48-4abb-bbf6-a5edc14da680 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:36:59.764509 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644dff18-af48-4abb-bbf6-a5edc14da680 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:36:59.764792 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644dff18-af48-4abb-bbf6-a5edc14da680 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:36:59.774161 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644dff18-af48-4abb-bbf6-a5edc14da680 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:36:59.776692 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644dff18-af48-4abb-bbf6-a5edc14da680 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:36:59.788611 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 790/1539] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:36:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:37:09.461223 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=32 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:37:09.783588 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=8 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:29.547382 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd06d1-5c3f-4a13-8a5f-c440b70e6d11 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:29.548022 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd06d1-5c3f-4a13-8a5f-c440b70e6d11 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:29.548022 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd06d1-5c3f-4a13-8a5f-c440b70e6d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:29.577826 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4fbd06d1-5c3f-4a13-8a5f-c440b70e6d11 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:29.579099 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4fbd06d1-5c3f-4a13-8a5f-c440b70e6d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:29.597607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 750/1540] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:29.607409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e1339-e66b-4a2f-9506-ab3c9ff7810e None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:29.608020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e1339-e66b-4a2f-9506-ab3c9ff7810e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:29.608020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e1339-e66b-4a2f-9506-ab3c9ff7810e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:29.639856 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-f94e1339-e66b-4a2f-9506-ab3c9ff7810e None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:29.641123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f94e1339-e66b-4a2f-9506-ab3c9ff7810e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:29.651019 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 791/1541] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:29.656445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd06d1-5c3f-4a13-8a5f-c440b70e6d11 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:29.673338 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd06d1-5c3f-4a13-8a5f-c440b70e6d11 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:29.681328 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d73966-16de-4b6e-86f0-b18ec2b58ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:29.681328 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d73966-16de-4b6e-86f0-b18ec2b58ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:29.681328 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d73966-16de-4b6e-86f0-b18ec2b58ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:29.681700 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d73966-16de-4b6e-86f0-b18ec2b58ab3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:29.682395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d73966-16de-4b6e-86f0-b18ec2b58ab3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:38:29.682582 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d73966-16de-4b6e-86f0-b18ec2b58ab3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:29.684193 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d73966-16de-4b6e-86f0-b18ec2b58ab3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:29.707532 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 751/1542] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:38:29.716751 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e1339-e66b-4a2f-9506-ab3c9ff7810e None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:29.728374 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e1339-e66b-4a2f-9506-ab3c9ff7810e None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:29.745954 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b057e8-a720-46fb-a25b-69548eed8978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:29.747043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b057e8-a720-46fb-a25b-69548eed8978 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:29.747343 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b057e8-a720-46fb-a25b-69548eed8978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:29.747626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b057e8-a720-46fb-a25b-69548eed8978 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:29.748374 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b057e8-a720-46fb-a25b-69548eed8978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:29.750426 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b057e8-a720-46fb-a25b-69548eed8978 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:29.800495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 792/1543] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:38:29 2022] POST /identity/v3/projects => generated 367 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:29.805302 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d73966-16de-4b6e-86f0-b18ec2b58ab3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:29.828588 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d73966-16de-4b6e-86f0-b18ec2b58ab3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:29.913107 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba74a7-5c39-487c-8fa5-29f0c5b249fc None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:29.913385 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba74a7-5c39-487c-8fa5-29f0c5b249fc None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:29.913757 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba74a7-5c39-487c-8fa5-29f0c5b249fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:29.914478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2d30b-0eb7-4183-95bc-e0477f2769c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:29.916021 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2d30b-0eb7-4183-95bc-e0477f2769c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:29.916592 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2d30b-0eb7-4183-95bc-e0477f2769c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:29.917401 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2d30b-0eb7-4183-95bc-e0477f2769c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:29.918883 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2d30b-0eb7-4183-95bc-e0477f2769c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:29.922449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2d30b-0eb7-4183-95bc-e0477f2769c2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:29.949959 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-46ba74a7-5c39-487c-8fa5-29f0c5b249fc None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:29.951422 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-46ba74a7-5c39-487c-8fa5-29f0c5b249fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:29.958597 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 752/1544] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:38:29 2022] POST /identity/v3/users => generated 284 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:29.962536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2d30b-0eb7-4183-95bc-e0477f2769c2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:29.968944 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 793/1545] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:29.979199 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc3cb5-74fa-44f3-a90b-1b50146b95b5 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:29.979691 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc3cb5-74fa-44f3-a90b-1b50146b95b5 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:29.979977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc3cb5-74fa-44f3-a90b-1b50146b95b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:29.981796 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2d30b-0eb7-4183-95bc-e0477f2769c2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:29.993894 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414053fc-8b0e-41c2-859c-dca39ed4f6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:29.995003 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414053fc-8b0e-41c2-859c-dca39ed4f6fd admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:29.996023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414053fc-8b0e-41c2-859c-dca39ed4f6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:29.996023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414053fc-8b0e-41c2-859c-dca39ed4f6fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:29.997079 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414053fc-8b0e-41c2-859c-dca39ed4f6fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:29.998991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414053fc-8b0e-41c2-859c-dca39ed4f6fd admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:30.010914 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a0fc3cb5-74fa-44f3-a90b-1b50146b95b5 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:30.012168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a0fc3cb5-74fa-44f3-a90b-1b50146b95b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:30.019345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 753/1546] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:30.023210 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414053fc-8b0e-41c2-859c-dca39ed4f6fd admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:30.036566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 794/1547] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:38:30.042107 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc3cb5-74fa-44f3-a90b-1b50146b95b5 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:30.043808 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414053fc-8b0e-41c2-859c-dca39ed4f6fd admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:30.057248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc3cb5-74fa-44f3-a90b-1b50146b95b5 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:30.064044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79d960-3682-45f6-9e64-4c5d34a515b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:30.065114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79d960-3682-45f6-9e64-4c5d34a515b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.065888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79d960-3682-45f6-9e64-4c5d34a515b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.066373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79d960-3682-45f6-9e64-4c5d34a515b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.067352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd79d960-3682-45f6-9e64-4c5d34a515b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:30.067696 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86268b9-e959-424f-a8f5-82cf3b2f07e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:30.068518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86268b9-e959-424f-a8f5-82cf3b2f07e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.068807 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86268b9-e959-424f-a8f5-82cf3b2f07e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.069226 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86268b9-e959-424f-a8f5-82cf3b2f07e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.070134 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86268b9-e959-424f-a8f5-82cf3b2f07e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:38:30.070364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd79d960-3682-45f6-9e64-4c5d34a515b6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:30.070476 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86268b9-e959-424f-a8f5-82cf3b2f07e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:30.071548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: WARNING py.warnings [None req-c86268b9-e959-424f-a8f5-82cf3b2f07e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 31 14:38:30.071548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: warnings.warn(msg) May 31 14:38:30.071548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: May 31 14:38:30.072427 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86268b9-e959-424f-a8f5-82cf3b2f07e3 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:30.090219 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 795/1548] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:38:30.092972 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 754/1549] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:30.096836 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86268b9-e959-424f-a8f5-82cf3b2f07e3 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:30.097463 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e9ecb-c91e-4919-856c-00d62f24002a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.097980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e9ecb-c91e-4919-856c-00d62f24002a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.098472 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e9ecb-c91e-4919-856c-00d62f24002a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.112742 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86268b9-e959-424f-a8f5-82cf3b2f07e3 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:30.137971 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe4e25-a6e5-4c58-bf09-249bf86644ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:30.138937 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe4e25-a6e5-4c58-bf09-249bf86644ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.139222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe4e25-a6e5-4c58-bf09-249bf86644ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.139489 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe4e25-a6e5-4c58-bf09-249bf86644ee admin admin] PATH_INFO: `/v3/projects/46aee1976e2542bfaaf81e1ee728a29d/users/b7259f3236b3440a8826eeae2d5f3458/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.139825 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6c4e9ecb-c91e-4919-856c-00d62f24002a None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:30.141271 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6c4e9ecb-c91e-4919-856c-00d62f24002a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:30.168813 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe4e25-a6e5-4c58-bf09-249bf86644ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=46aee1976e2542bfaaf81e1ee728a29d, user_id=b7259f3236b3440a8826eeae2d5f3458, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:30.171767 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe4e25-a6e5-4c58-bf09-249bf86644ee admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:30.172341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 755/1550] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:30.173372 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e9ecb-c91e-4919-856c-00d62f24002a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:30.189588 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e9ecb-c91e-4919-856c-00d62f24002a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:30.201530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 796/1551] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:30 2022] PUT /identity/v3/projects/46aee1976e2542bfaaf81e1ee728a29d/users/b7259f3236b3440a8826eeae2d5f3458/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:30.202875 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3caa2d8-45e0-4585-92b6-da65566ce96f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.203169 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3caa2d8-45e0-4585-92b6-da65566ce96f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.203449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3caa2d8-45e0-4585-92b6-da65566ce96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.206155 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2ce48-7222-4f19-8c8c-6d5ae3aed3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:30.206865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2ce48-7222-4f19-8c8c-6d5ae3aed3c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.207151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2ce48-7222-4f19-8c8c-6d5ae3aed3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.207426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2ce48-7222-4f19-8c8c-6d5ae3aed3c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.208134 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2ce48-7222-4f19-8c8c-6d5ae3aed3c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:30.209164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-41d2ce48-7222-4f19-8c8c-6d5ae3aed3c8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 31 14:38:30.209164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: warnings.warn(msg) May 31 14:38:30.209164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: May 31 14:38:30.210903 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2ce48-7222-4f19-8c8c-6d5ae3aed3c8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:30.235397 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-e3caa2d8-45e0-4585-92b6-da65566ce96f None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:30.238176 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e3caa2d8-45e0-4585-92b6-da65566ce96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:30.254769 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 756/1552] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:38:30 2022] POST /identity/v3/projects => generated 391 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:30.256036 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601cf58-2264-4d01-8732-b96149281f15 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.256346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601cf58-2264-4d01-8732-b96149281f15 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.256701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601cf58-2264-4d01-8732-b96149281f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.275202 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a601cf58-2264-4d01-8732-b96149281f15 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:30.276420 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a601cf58-2264-4d01-8732-b96149281f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:30.312102 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 797/1553] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:30.313361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1983e00-11c6-4d44-9db7-325d05fb8c68 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.313794 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1983e00-11c6-4d44-9db7-325d05fb8c68 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.314112 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1983e00-11c6-4d44-9db7-325d05fb8c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.333266 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a1983e00-11c6-4d44-9db7-325d05fb8c68 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:30.334505 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a1983e00-11c6-4d44-9db7-325d05fb8c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:30.344605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 757/1554] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:30.344605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26b4a5-b487-4a7a-8712-d192c2895dba None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.344605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26b4a5-b487-4a7a-8712-d192c2895dba None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.344605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26b4a5-b487-4a7a-8712-d192c2895dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.349547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 798/1555] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:30.349547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4b5fe-de40-4b7d-9da6-e37e5cfb50cb None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.349547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4b5fe-de40-4b7d-9da6-e37e5cfb50cb None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.349547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4b5fe-de40-4b7d-9da6-e37e5cfb50cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.365385 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-63e4b5fe-de40-4b7d-9da6-e37e5cfb50cb None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:30.366585 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-63e4b5fe-de40-4b7d-9da6-e37e5cfb50cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:30.369321 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5e26b4a5-b487-4a7a-8712-d192c2895dba None None] MFA Rules not processed for user `b7259f3236b3440a8826eeae2d5f3458`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:30.370555 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5e26b4a5-b487-4a7a-8712-d192c2895dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:30.374901 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 799/1556] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:30.376113 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac51a65-167c-41a8-9c87-5d4792e26677 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.376399 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac51a65-167c-41a8-9c87-5d4792e26677 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.376677 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac51a65-167c-41a8-9c87-5d4792e26677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.386499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:30.394553 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-2ac51a65-167c-41a8-9c87-5d4792e26677 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:30.396102 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2ac51a65-167c-41a8-9c87-5d4792e26677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:30.404016 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 800/1557] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:30.404855 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac51a65-167c-41a8-9c87-5d4792e26677 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:30.414026 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac51a65-167c-41a8-9c87-5d4792e26677 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:30.418528 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f14fa-81d0-4072-a79e-bc0a17fc8719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:30.419203 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f14fa-81d0-4072-a79e-bc0a17fc8719 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.419489 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f14fa-81d0-4072-a79e-bc0a17fc8719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.419763 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f14fa-81d0-4072-a79e-bc0a17fc8719 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.420439 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f14fa-81d0-4072-a79e-bc0a17fc8719 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:30.421456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: WARNING py.warnings [None req-2a5f14fa-81d0-4072-a79e-bc0a17fc8719 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 31 14:38:30.421456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: warnings.warn(msg) May 31 14:38:30.421456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: May 31 14:38:30.422636 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f14fa-81d0-4072-a79e-bc0a17fc8719 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:30.443955 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 801/1558] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:38:30 2022] POST /identity/v3/users => generated 296 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:30.445191 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 758/1559] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:30 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:30.446905 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf421a-04c1-4635-ad4b-80c834a9124d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.447203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf421a-04c1-4635-ad4b-80c834a9124d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.447648 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf421a-04c1-4635-ad4b-80c834a9124d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.449712 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935939e-5357-4dd6-b961-4a92c60ed890 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.450022 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935939e-5357-4dd6-b961-4a92c60ed890 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.450313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935939e-5357-4dd6-b961-4a92c60ed890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.465992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-dcdf421a-04c1-4635-ad4b-80c834a9124d None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:30.467258 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dcdf421a-04c1-4635-ad4b-80c834a9124d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:30.468792 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-2935939e-5357-4dd6-b961-4a92c60ed890 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:30.469957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2935939e-5357-4dd6-b961-4a92c60ed890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:30.475201 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 759/1560] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:30.476415 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819275d-bff5-4cff-ae2a-b3963a3eb2cb None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.476703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819275d-bff5-4cff-ae2a-b3963a3eb2cb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.476983 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819275d-bff5-4cff-ae2a-b3963a3eb2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.485281 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 802/1561] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:30.485281 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935939e-5357-4dd6-b961-4a92c60ed890 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:30.495784 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935939e-5357-4dd6-b961-4a92c60ed890 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:30.496208 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f819275d-bff5-4cff-ae2a-b3963a3eb2cb None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:30.497406 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f819275d-bff5-4cff-ae2a-b3963a3eb2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:30.501846 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd702b3-1146-4d5d-a4cf-59ad6979690e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:30.502504 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd702b3-1146-4d5d-a4cf-59ad6979690e admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.502830 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd702b3-1146-4d5d-a4cf-59ad6979690e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.503111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd702b3-1146-4d5d-a4cf-59ad6979690e admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.503837 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd702b3-1146-4d5d-a4cf-59ad6979690e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:38:30.504146 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd702b3-1146-4d5d-a4cf-59ad6979690e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:30.505343 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 760/1562] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:30.509816 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd702b3-1146-4d5d-a4cf-59ad6979690e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:30.509949 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f90052-7488-4a93-ac62-4c21d9144d37 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.509949 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f90052-7488-4a93-ac62-4c21d9144d37 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.509949 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f90052-7488-4a93-ac62-4c21d9144d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.520304 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 803/1563] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:38:30.521598 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83b9fa-b29b-4cea-96b2-3255aae0ddbf None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.521881 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83b9fa-b29b-4cea-96b2-3255aae0ddbf None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.522164 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83b9fa-b29b-4cea-96b2-3255aae0ddbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.528373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-72f90052-7488-4a93-ac62-4c21d9144d37 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:30.529703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-72f90052-7488-4a93-ac62-4c21d9144d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:30.537855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 761/1564] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:30.538639 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f90052-7488-4a93-ac62-4c21d9144d37 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:30.547720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5b83b9fa-b29b-4cea-96b2-3255aae0ddbf None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:30.548348 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f90052-7488-4a93-ac62-4c21d9144d37 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:30.548910 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5b83b9fa-b29b-4cea-96b2-3255aae0ddbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:30.553029 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece903c-a4b3-4768-b4fa-a537c3e615df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:30.553685 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece903c-a4b3-4768-b4fa-a537c3e615df admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.553973 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece903c-a4b3-4768-b4fa-a537c3e615df admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.554257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece903c-a4b3-4768-b4fa-a537c3e615df admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.555008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece903c-a4b3-4768-b4fa-a537c3e615df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:30.557303 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece903c-a4b3-4768-b4fa-a537c3e615df admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:30.557412 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 804/1565] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:30.558068 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b83b9fa-b29b-4cea-96b2-3255aae0ddbf None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:30.568056 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 762/1566] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:30.568682 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece903c-a4b3-4768-b4fa-a537c3e615df admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:30.580507 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:30.582761 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece903c-a4b3-4768-b4fa-a537c3e615df admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:30.587436 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa714b-faf6-42ac-8cb9-caee583424bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:30.588159 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa714b-faf6-42ac-8cb9-caee583424bf admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.588454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa714b-faf6-42ac-8cb9-caee583424bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.588737 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa714b-faf6-42ac-8cb9-caee583424bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.589465 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa714b-faf6-42ac-8cb9-caee583424bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:38:30.589778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa714b-faf6-42ac-8cb9-caee583424bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:30.591418 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa714b-faf6-42ac-8cb9-caee583424bf admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:30.605371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 763/1567] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:38:30.606803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8559c6a-7573-49ea-96b6-8a2ac1b0ac05 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.607091 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8559c6a-7573-49ea-96b6-8a2ac1b0ac05 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.607364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8559c6a-7573-49ea-96b6-8a2ac1b0ac05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.627224 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b83b9fa-b29b-4cea-96b2-3255aae0ddbf None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:30.628912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b8559c6a-7573-49ea-96b6-8a2ac1b0ac05 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:30.630115 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b8559c6a-7573-49ea-96b6-8a2ac1b0ac05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:30.631102 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc90ea1-9030-470f-801b-2ffaae40bcd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:30.631757 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc90ea1-9030-470f-801b-2ffaae40bcd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.632042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc90ea1-9030-470f-801b-2ffaae40bcd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.632345 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc90ea1-9030-470f-801b-2ffaae40bcd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.638358 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 764/1568] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:30.639163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8559c6a-7573-49ea-96b6-8a2ac1b0ac05 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:30.644710 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc90ea1-9030-470f-801b-2ffaae40bcd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:30.646148 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc90ea1-9030-470f-801b-2ffaae40bcd3 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:30.648110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8559c6a-7573-49ea-96b6-8a2ac1b0ac05 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:30.652635 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fda676-40b6-40fd-ad45-4168a1a75e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:30.653272 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fda676-40b6-40fd-ad45-4168a1a75e30 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.653562 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fda676-40b6-40fd-ad45-4168a1a75e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.653836 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fda676-40b6-40fd-ad45-4168a1a75e30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.654546 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fda676-40b6-40fd-ad45-4168a1a75e30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:38:30.654883 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fda676-40b6-40fd-ad45-4168a1a75e30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:30.655233 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 805/1569] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:30 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:30.655991 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc90ea1-9030-470f-801b-2ffaae40bcd3 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:30.656489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fda676-40b6-40fd-ad45-4168a1a75e30 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:30.666377 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc90ea1-9030-470f-801b-2ffaae40bcd3 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:30.676950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 765/1570] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:38:30.676950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fda676-40b6-40fd-ad45-4168a1a75e30 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:30.678695 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c164f-bb71-4eba-afef-3cdc16bc26c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:30.679508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c164f-bb71-4eba-afef-3cdc16bc26c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.679791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c164f-bb71-4eba-afef-3cdc16bc26c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.680087 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c164f-bb71-4eba-afef-3cdc16bc26c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.681507 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c164f-bb71-4eba-afef-3cdc16bc26c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:30.683333 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c164f-bb71-4eba-afef-3cdc16bc26c3 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:30.688857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fda676-40b6-40fd-ad45-4168a1a75e30 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:30.693644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a905d-2921-4364-a5ba-51a760e08022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:30.694365 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a905d-2921-4364-a5ba-51a760e08022 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.694668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a905d-2921-4364-a5ba-51a760e08022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.694955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a905d-2921-4364-a5ba-51a760e08022 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.695675 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a905d-2921-4364-a5ba-51a760e08022 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:38:30.695982 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a905d-2921-4364-a5ba-51a760e08022 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:30.697621 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a905d-2921-4364-a5ba-51a760e08022 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:30.710912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 766/1571] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:38:30.712036 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a905d-2921-4364-a5ba-51a760e08022 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:30.716053 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 806/1572] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:38:30 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:30.717601 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c164f-bb71-4eba-afef-3cdc16bc26c3 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:30.721999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a905d-2921-4364-a5ba-51a760e08022 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:30.788770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6d9d6-65e3-4e85-9ce8-3a21adc72b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:30.789636 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6d9d6-65e3-4e85-9ce8-3a21adc72b81 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.790583 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6d9d6-65e3-4e85-9ce8-3a21adc72b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.790979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6d9d6-65e3-4e85-9ce8-3a21adc72b81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.792180 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6d9d6-65e3-4e85-9ce8-3a21adc72b81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:38:30.792504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6d9d6-65e3-4e85-9ce8-3a21adc72b81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:30.794109 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6d9d6-65e3-4e85-9ce8-3a21adc72b81 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:30.797379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c164f-bb71-4eba-afef-3cdc16bc26c3 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:30.802186 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c4ecd-efc4-4791-b53a-1c227cdff83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:30.802898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c4ecd-efc4-4791-b53a-1c227cdff83e admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.803184 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c4ecd-efc4-4791-b53a-1c227cdff83e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.803465 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c4ecd-efc4-4791-b53a-1c227cdff83e admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.804205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c4ecd-efc4-4791-b53a-1c227cdff83e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:30.806432 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c4ecd-efc4-4791-b53a-1c227cdff83e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:30.812467 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 767/1573] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:38:30.812467 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6d9d6-65e3-4e85-9ce8-3a21adc72b81 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:30.823964 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 807/1574] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:30.824770 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c4ecd-efc4-4791-b53a-1c227cdff83e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:30.826079 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6d9d6-65e3-4e85-9ce8-3a21adc72b81 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:30.832105 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44682201-91de-47f8-9969-43696b98b330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:30.833009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44682201-91de-47f8-9969-43696b98b330 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.833365 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44682201-91de-47f8-9969-43696b98b330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.833714 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44682201-91de-47f8-9969-43696b98b330 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.834477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44682201-91de-47f8-9969-43696b98b330 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:30.835066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c4ecd-efc4-4791-b53a-1c227cdff83e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:30.837152 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44682201-91de-47f8-9969-43696b98b330 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:30.839928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e4d40-9ffa-47c6-909a-5413e80bc7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:30.840919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e4d40-9ffa-47c6-909a-5413e80bc7de admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.841048 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e4d40-9ffa-47c6-909a-5413e80bc7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.841300 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e4d40-9ffa-47c6-909a-5413e80bc7de admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.842033 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e4d40-9ffa-47c6-909a-5413e80bc7de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:38:30.842619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e4d40-9ffa-47c6-909a-5413e80bc7de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:30.844032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e4d40-9ffa-47c6-909a-5413e80bc7de admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:30.857173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 808/1575] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:38:30.858158 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e4d40-9ffa-47c6-909a-5413e80bc7de admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:30.870269 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e4d40-9ffa-47c6-909a-5413e80bc7de admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:30.875282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea6d89-6f61-41d9-9e0c-94f3cddd4840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:30.875951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea6d89-6f61-41d9-9e0c-94f3cddd4840 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:30.876287 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea6d89-6f61-41d9-9e0c-94f3cddd4840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:30.876545 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea6d89-6f61-41d9-9e0c-94f3cddd4840 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:30.877246 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea6d89-6f61-41d9-9e0c-94f3cddd4840 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:30.877980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 768/1576] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:38:30 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:30.878933 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44682201-91de-47f8-9969-43696b98b330 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:30.879835 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea6d89-6f61-41d9-9e0c-94f3cddd4840 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:30.885627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:30.911367 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 809/1577] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:38:30 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:30.915692 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea6d89-6f61-41d9-9e0c-94f3cddd4840 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:30.984057 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44682201-91de-47f8-9969-43696b98b330 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.008511 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea6d89-6f61-41d9-9e0c-94f3cddd4840 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.014726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40728604-a80c-48c3-ab8c-f1a603e08d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.015521 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40728604-a80c-48c3-ab8c-f1a603e08d05 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.015810 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40728604-a80c-48c3-ab8c-f1a603e08d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.016088 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40728604-a80c-48c3-ab8c-f1a603e08d05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.016780 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40728604-a80c-48c3-ab8c-f1a603e08d05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.018834 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40728604-a80c-48c3-ab8c-f1a603e08d05 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.022167 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:31.044043 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c2d0e-19db-420a-b40a-ca55d0d60a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.044726 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c2d0e-19db-420a-b40a-ca55d0d60a20 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.045014 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c2d0e-19db-420a-b40a-ca55d0d60a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.045296 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c2d0e-19db-420a-b40a-ca55d0d60a20 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.045989 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322c2d0e-19db-420a-b40a-ca55d0d60a20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.047490 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 810/1578] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:38:30 2022] POST /identity/v3/projects => generated 373 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:31.047948 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322c2d0e-19db-420a-b40a-ca55d0d60a20 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.048308 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40728604-a80c-48c3-ab8c-f1a603e08d05 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.072174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 769/1579] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:38:30 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:31.073071 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c2d0e-19db-420a-b40a-ca55d0d60a20 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.085241 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c2d0e-19db-420a-b40a-ca55d0d60a20 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.151057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40728604-a80c-48c3-ab8c-f1a603e08d05 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.157163 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d75b864-1590-4ec7-8cdd-3da4a18e85d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.157963 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75b864-1590-4ec7-8cdd-3da4a18e85d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.158248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75b864-1590-4ec7-8cdd-3da4a18e85d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.158501 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075931e3-1c43-4746-9e83-32ab670f6f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.158782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75b864-1590-4ec7-8cdd-3da4a18e85d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.159491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d75b864-1590-4ec7-8cdd-3da4a18e85d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.159597 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075931e3-1c43-4746-9e83-32ab670f6f9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.159837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075931e3-1c43-4746-9e83-32ab670f6f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.160122 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075931e3-1c43-4746-9e83-32ab670f6f9a admin admin] PATH_INFO: `/v3/projects/8c940479079b46cf82136117f62c3f1a/users/91255c0b2d914ef9be10eb69d7d8ed4f/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.161428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d75b864-1590-4ec7-8cdd-3da4a18e85d2 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.180895 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075931e3-1c43-4746-9e83-32ab670f6f9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c940479079b46cf82136117f62c3f1a, user_id=91255c0b2d914ef9be10eb69d7d8ed4f, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.182031 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-075931e3-1c43-4746-9e83-32ab670f6f9a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 31 14:38:31.182031 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: warnings.warn(msg) May 31 14:38:31.182031 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: May 31 14:38:31.184089 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075931e3-1c43-4746-9e83-32ab670f6f9a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.199016 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 770/1580] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:31 2022] PUT /identity/v3/projects/8c940479079b46cf82136117f62c3f1a/users/91255c0b2d914ef9be10eb69d7d8ed4f/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:31.203442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075931e3-1c43-4746-9e83-32ab670f6f9a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.204461 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 811/1581] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:38:31 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:31.205512 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d75b864-1590-4ec7-8cdd-3da4a18e85d2 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.249102 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:31.273250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d75b864-1590-4ec7-8cdd-3da4a18e85d2 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.279834 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943dce43-c289-4761-9a7b-b67e8c293d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.280520 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943dce43-c289-4761-9a7b-b67e8c293d7f admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.280808 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943dce43-c289-4761-9a7b-b67e8c293d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.281089 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943dce43-c289-4761-9a7b-b67e8c293d7f admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.281848 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943dce43-c289-4761-9a7b-b67e8c293d7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.283818 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943dce43-c289-4761-9a7b-b67e8c293d7f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.303979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075931e3-1c43-4746-9e83-32ab670f6f9a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.311140 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5763b-d3c1-491f-a823-aff28ced27dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.311679 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 812/1582] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:38:31 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:31.312222 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5763b-d3c1-491f-a823-aff28ced27dc admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.312506 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943dce43-c289-4761-9a7b-b67e8c293d7f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.312849 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5763b-d3c1-491f-a823-aff28ced27dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.313309 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5763b-d3c1-491f-a823-aff28ced27dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.314206 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5763b-d3c1-491f-a823-aff28ced27dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.316839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5763b-d3c1-491f-a823-aff28ced27dc admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.327079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943dce43-c289-4761-9a7b-b67e8c293d7f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.331986 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b4f23-0716-491b-9cba-f5dc8ba2f025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.332762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b4f23-0716-491b-9cba-f5dc8ba2f025 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.333048 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b4f23-0716-491b-9cba-f5dc8ba2f025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.333328 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b4f23-0716-491b-9cba-f5dc8ba2f025 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.334024 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b4f23-0716-491b-9cba-f5dc8ba2f025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.336095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b4f23-0716-491b-9cba-f5dc8ba2f025 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.351334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 771/1583] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:38:31 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:31.352711 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5763b-d3c1-491f-a823-aff28ced27dc admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.361314 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 813/1584] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:38:31 2022] POST /identity/v3/users => generated 283 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:31.362251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b4f23-0716-491b-9cba-f5dc8ba2f025 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.373928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b4f23-0716-491b-9cba-f5dc8ba2f025 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.421445 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:31.424990 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5763b-d3c1-491f-a823-aff28ced27dc admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.430229 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2203f-6355-4c45-8f4b-5f6804bca67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.430974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2203f-6355-4c45-8f4b-5f6804bca67e admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.431260 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2203f-6355-4c45-8f4b-5f6804bca67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.431652 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2203f-6355-4c45-8f4b-5f6804bca67e admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.432346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2203f-6355-4c45-8f4b-5f6804bca67e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.434370 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2203f-6355-4c45-8f4b-5f6804bca67e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.438148 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233bbbdf-9ce1-4fda-9e01-d36a046fc02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.439041 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bbbdf-9ce1-4fda-9e01-d36a046fc02a admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.439404 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bbbdf-9ce1-4fda-9e01-d36a046fc02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.439783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bbbdf-9ce1-4fda-9e01-d36a046fc02a admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.440882 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233bbbdf-9ce1-4fda-9e01-d36a046fc02a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.442661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233bbbdf-9ce1-4fda-9e01-d36a046fc02a admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.453577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 814/1585] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:31.454879 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f0154-6297-4322-9620-fae0e08cf8ad None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.455345 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f0154-6297-4322-9620-fae0e08cf8ad None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.455635 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f0154-6297-4322-9620-fae0e08cf8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.457485 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 772/1586] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:38:31 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:31.458314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2203f-6355-4c45-8f4b-5f6804bca67e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.468949 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2203f-6355-4c45-8f4b-5f6804bca67e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.474105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-186f0154-6297-4322-9620-fae0e08cf8ad None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:31.475334 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-186f0154-6297-4322-9620-fae0e08cf8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:31.476423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2dcd1b-6c53-4bf5-8659-d6a7a0f4f813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.477095 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dcd1b-6c53-4bf5-8659-d6a7a0f4f813 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.477438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dcd1b-6c53-4bf5-8659-d6a7a0f4f813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.477733 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dcd1b-6c53-4bf5-8659-d6a7a0f4f813 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.478426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2dcd1b-6c53-4bf5-8659-d6a7a0f4f813 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.480391 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2dcd1b-6c53-4bf5-8659-d6a7a0f4f813 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.483255 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 815/1587] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:31.487171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f0154-6297-4322-9620-fae0e08cf8ad None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.497345 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f0154-6297-4322-9620-fae0e08cf8ad None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.500813 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 773/1588] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:38:31 2022] POST /identity/v3/users => generated 287 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:31.501580 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2dcd1b-6c53-4bf5-8659-d6a7a0f4f813 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.504949 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164fbe7-cae2-48bc-a9e2-acf17e015338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.505618 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164fbe7-cae2-48bc-a9e2-acf17e015338 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.505904 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164fbe7-cae2-48bc-a9e2-acf17e015338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.506187 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164fbe7-cae2-48bc-a9e2-acf17e015338 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.506957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164fbe7-cae2-48bc-a9e2-acf17e015338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.508563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164fbe7-cae2-48bc-a9e2-acf17e015338 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.511716 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2dcd1b-6c53-4bf5-8659-d6a7a0f4f813 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.516619 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4446c1-aaf7-4b80-96fb-920d494ae426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.517277 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4446c1-aaf7-4b80-96fb-920d494ae426 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.517561 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4446c1-aaf7-4b80-96fb-920d494ae426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.517844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4446c1-aaf7-4b80-96fb-920d494ae426 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.518525 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4446c1-aaf7-4b80-96fb-920d494ae426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.522697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 816/1589] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:31.522831 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4446c1-aaf7-4b80-96fb-920d494ae426 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.522938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164fbe7-cae2-48bc-a9e2-acf17e015338 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.531056 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164fbe7-cae2-48bc-a9e2-acf17e015338 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.536097 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f310ee7-1956-416e-a9b5-02b683089009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.536761 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f310ee7-1956-416e-a9b5-02b683089009 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.537055 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f310ee7-1956-416e-a9b5-02b683089009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.537333 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f310ee7-1956-416e-a9b5-02b683089009 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.538061 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f310ee7-1956-416e-a9b5-02b683089009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.539696 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f310ee7-1956-416e-a9b5-02b683089009 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.542199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 774/1590] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:38:31 2022] POST /identity/v3/users => generated 289 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:31.543241 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4446c1-aaf7-4b80-96fb-920d494ae426 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.552667 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4446c1-aaf7-4b80-96fb-920d494ae426 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.553319 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 817/1591] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:31.556532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f310ee7-1956-416e-a9b5-02b683089009 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.557965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe73db-23d5-4904-9845-711b971994d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.560337 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe73db-23d5-4904-9845-711b971994d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.560337 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe73db-23d5-4904-9845-711b971994d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.560337 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe73db-23d5-4904-9845-711b971994d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.560337 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe73db-23d5-4904-9845-711b971994d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.561798 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe73db-23d5-4904-9845-711b971994d0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.565357 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f310ee7-1956-416e-a9b5-02b683089009 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.570342 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf17ee4-cbfb-450a-94e7-96dca07f8b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.571025 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf17ee4-cbfb-450a-94e7-96dca07f8b9b admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.571844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf17ee4-cbfb-450a-94e7-96dca07f8b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.572166 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf17ee4-cbfb-450a-94e7-96dca07f8b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.572847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf17ee4-cbfb-450a-94e7-96dca07f8b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.573021 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 775/1592] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:31.573905 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe73db-23d5-4904-9845-711b971994d0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.574441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf17ee4-cbfb-450a-94e7-96dca07f8b9b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.589170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 818/1593] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:31.589170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf17ee4-cbfb-450a-94e7-96dca07f8b9b admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.597481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf17ee4-cbfb-450a-94e7-96dca07f8b9b admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.602772 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4ce28-cd85-476d-a690-7532b0cfdc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.603799 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4ce28-cd85-476d-a690-7532b0cfdc45 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.604087 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4ce28-cd85-476d-a690-7532b0cfdc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.604365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4ce28-cd85-476d-a690-7532b0cfdc45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.605087 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a4ce28-cd85-476d-a690-7532b0cfdc45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.606673 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a4ce28-cd85-476d-a690-7532b0cfdc45 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.617452 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 819/1594] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:31.618827 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4ce28-cd85-476d-a690-7532b0cfdc45 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.630315 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe73db-23d5-4904-9845-711b971994d0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.630482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4ce28-cd85-476d-a690-7532b0cfdc45 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.634373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b13e8-847a-405f-9ed5-a91af3a48115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.635051 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b13e8-847a-405f-9ed5-a91af3a48115 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.635334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b13e8-847a-405f-9ed5-a91af3a48115 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.635519 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e0e7b-4b0c-4dfd-b163-f952dfd586a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.635674 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6b13e8-847a-405f-9ed5-a91af3a48115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.636155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e0e7b-4b0c-4dfd-b163-f952dfd586a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.636441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e0e7b-4b0c-4dfd-b163-f952dfd586a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.636720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e0e7b-4b0c-4dfd-b163-f952dfd586a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.637447 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e0e7b-4b0c-4dfd-b163-f952dfd586a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.639320 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e0e7b-4b0c-4dfd-b163-f952dfd586a8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.643168 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:31.645910 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6b13e8-847a-405f-9ed5-a91af3a48115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.647474 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6b13e8-847a-405f-9ed5-a91af3a48115 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.649868 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 820/1595] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:31.653331 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e0e7b-4b0c-4dfd-b163-f952dfd586a8 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.661634 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 776/1596] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:31.662866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e0e7b-4b0c-4dfd-b163-f952dfd586a8 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.663171 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b13e8-847a-405f-9ed5-a91af3a48115 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.668084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6aa35a-7fee-4137-b411-ae19af6a80ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.669192 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6aa35a-7fee-4137-b411-ae19af6a80ad admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.669192 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6aa35a-7fee-4137-b411-ae19af6a80ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.669602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6aa35a-7fee-4137-b411-ae19af6a80ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.670512 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6aa35a-7fee-4137-b411-ae19af6a80ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.672428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6aa35a-7fee-4137-b411-ae19af6a80ad admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.672610 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6b13e8-847a-405f-9ed5-a91af3a48115 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.677157 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de8fb2-0423-4ba4-90dd-e1b8720c0eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.677812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de8fb2-0423-4ba4-90dd-e1b8720c0eae admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.678098 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de8fb2-0423-4ba4-90dd-e1b8720c0eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.678424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de8fb2-0423-4ba4-90dd-e1b8720c0eae admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.679126 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de8fb2-0423-4ba4-90dd-e1b8720c0eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.681248 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de8fb2-0423-4ba4-90dd-e1b8720c0eae admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.684965 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 821/1597] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:31.688752 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6aa35a-7fee-4137-b411-ae19af6a80ad admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.696105 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 777/1598] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:31.696105 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de8fb2-0423-4ba4-90dd-e1b8720c0eae admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.700922 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6aa35a-7fee-4137-b411-ae19af6a80ad admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.705639 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e436df6-0182-4a66-a112-8048c83030d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.706514 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e436df6-0182-4a66-a112-8048c83030d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.706820 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e436df6-0182-4a66-a112-8048c83030d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.707096 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e436df6-0182-4a66-a112-8048c83030d4 admin admin] PATH_INFO: `/v3/projects/0300307fde534d49a050041217cfe6e1/users/ea126c816da740dda9622f7bfa10dd16/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.708217 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de8fb2-0423-4ba4-90dd-e1b8720c0eae admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.714390 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b44982-c174-4318-b6ba-a56ca4c36745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.714944 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b44982-c174-4318-b6ba-a56ca4c36745 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.715263 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b44982-c174-4318-b6ba-a56ca4c36745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.715554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b44982-c174-4318-b6ba-a56ca4c36745 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.716413 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b44982-c174-4318-b6ba-a56ca4c36745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.718179 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b44982-c174-4318-b6ba-a56ca4c36745 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.722301 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e436df6-0182-4a66-a112-8048c83030d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0300307fde534d49a050041217cfe6e1, user_id=ea126c816da740dda9622f7bfa10dd16, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.725409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e436df6-0182-4a66-a112-8048c83030d4 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.733846 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 778/1599] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:31.733846 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b44982-c174-4318-b6ba-a56ca4c36745 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.744899 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b44982-c174-4318-b6ba-a56ca4c36745 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.747062 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 822/1600] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:31 2022] PUT /identity/v3/projects/0300307fde534d49a050041217cfe6e1/users/ea126c816da740dda9622f7bfa10dd16/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:31.748021 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e436df6-0182-4a66-a112-8048c83030d4 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.811723 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e4de6-3d0b-4ddb-9bb3-5f949dc761f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.812599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e4de6-3d0b-4ddb-9bb3-5f949dc761f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.813008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e4de6-3d0b-4ddb-9bb3-5f949dc761f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.813638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e4de6-3d0b-4ddb-9bb3-5f949dc761f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.814521 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e4de6-3d0b-4ddb-9bb3-5f949dc761f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.818053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e4de6-3d0b-4ddb-9bb3-5f949dc761f3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.820331 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e436df6-0182-4a66-a112-8048c83030d4 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.827292 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ade798-dd86-463a-979a-9abba42aa6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.828263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ade798-dd86-463a-979a-9abba42aa6e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.828599 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ade798-dd86-463a-979a-9abba42aa6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.828886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ade798-dd86-463a-979a-9abba42aa6e3 admin admin] PATH_INFO: `/v3/projects/2ebca4039f664206a5e793438f28b98f/users/2c73c11b11c74540bb46302363604730/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.830564 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 779/1601] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:31.831474 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e4de6-3d0b-4ddb-9bb3-5f949dc761f3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.844484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e4de6-3d0b-4ddb-9bb3-5f949dc761f3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.848515 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ade798-dd86-463a-979a-9abba42aa6e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ebca4039f664206a5e793438f28b98f, user_id=2c73c11b11c74540bb46302363604730, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.849793 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5f608-500f-4380-8a35-d453d2a91793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.850608 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5f608-500f-4380-8a35-d453d2a91793 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.851033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5f608-500f-4380-8a35-d453d2a91793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.851420 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5f608-500f-4380-8a35-d453d2a91793 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.851522 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ade798-dd86-463a-979a-9abba42aa6e3 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.852363 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe5f608-500f-4380-8a35-d453d2a91793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.855664 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe5f608-500f-4380-8a35-d453d2a91793 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.866866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 780/1602] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:31.868540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5f608-500f-4380-8a35-d453d2a91793 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.878233 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 823/1603] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:31 2022] PUT /identity/v3/projects/2ebca4039f664206a5e793438f28b98f/users/2c73c11b11c74540bb46302363604730/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:31.880956 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5f608-500f-4380-8a35-d453d2a91793 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.884405 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ade798-dd86-463a-979a-9abba42aa6e3 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:31.920709 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:31.952873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb83437-916d-40c3-a0f4-82a1f045f72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.954066 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb83437-916d-40c3-a0f4-82a1f045f72c admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.954347 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb83437-916d-40c3-a0f4-82a1f045f72c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.954623 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb83437-916d-40c3-a0f4-82a1f045f72c admin admin] PATH_INFO: `/v3/projects/91cafdeef693436ebfd7ad38408fe5ba/users/7ca6ef34e5624f868540d89c2a650e7b/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.963246 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ade798-dd86-463a-979a-9abba42aa6e3 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:31.968318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c8b26-cdd7-47ee-a630-a0b67509274c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:31.969359 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c8b26-cdd7-47ee-a630-a0b67509274c admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:31.969569 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c8b26-cdd7-47ee-a630-a0b67509274c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:31.969829 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c8b26-cdd7-47ee-a630-a0b67509274c admin admin] PATH_INFO: `/v3/projects/37b130e093da409daa4f34baccfc96cd/users/ae50190c609c4e47ba9371410a07d867/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:31.972932 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb83437-916d-40c3-a0f4-82a1f045f72c admin admin] RBAC: Authorizing `identity:create_grant(project_id=91cafdeef693436ebfd7ad38408fe5ba, user_id=7ca6ef34e5624f868540d89c2a650e7b, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.975949 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb83437-916d-40c3-a0f4-82a1f045f72c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.988150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c8b26-cdd7-47ee-a630-a0b67509274c admin admin] RBAC: Authorizing `identity:create_grant(project_id=37b130e093da409daa4f34baccfc96cd, user_id=ae50190c609c4e47ba9371410a07d867, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:31.990931 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c8b26-cdd7-47ee-a630-a0b67509274c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:31.994786 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 781/1604] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:31 2022] PUT /identity/v3/projects/91cafdeef693436ebfd7ad38408fe5ba/users/7ca6ef34e5624f868540d89c2a650e7b/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:32.000098 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d226f0-0d88-425e-908b-82e13d84d4e6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:32.000837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d226f0-0d88-425e-908b-82e13d84d4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:32.000837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d226f0-0d88-425e-908b-82e13d84d4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:32.015590 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 824/1605] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:31 2022] PUT /identity/v3/projects/37b130e093da409daa4f34baccfc96cd/users/ae50190c609c4e47ba9371410a07d867/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:32.023116 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c8b26-cdd7-47ee-a630-a0b67509274c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:32.025529 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b5d226f0-0d88-425e-908b-82e13d84d4e6 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:32.027441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b5d226f0-0d88-425e-908b-82e13d84d4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:32.085306 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:32.097236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c8b26-cdd7-47ee-a630-a0b67509274c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:32.103341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45b32c-cba4-4d8f-b10c-5146889095aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:32.103875 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45b32c-cba4-4d8f-b10c-5146889095aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:32.104159 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45b32c-cba4-4d8f-b10c-5146889095aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:32.104437 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45b32c-cba4-4d8f-b10c-5146889095aa admin admin] PATH_INFO: `/v3/projects/0849b36882564f989dfb96fd06203e09/users/2e14d892eae64aea8ef952d23cccdff3/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:32.120643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 782/1606] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:32.120643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d226f0-0d88-425e-908b-82e13d84d4e6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:32.129242 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45b32c-cba4-4d8f-b10c-5146889095aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=0849b36882564f989dfb96fd06203e09, user_id=2e14d892eae64aea8ef952d23cccdff3, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:32.133662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45b32c-cba4-4d8f-b10c-5146889095aa admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:32.142611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d226f0-0d88-425e-908b-82e13d84d4e6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:32.149290 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 825/1607] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:32 2022] PUT /identity/v3/projects/0849b36882564f989dfb96fd06203e09/users/2e14d892eae64aea8ef952d23cccdff3/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:32.150452 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43836cdc-682d-4991-8351-70b9ad369db1 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:32.150779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43836cdc-682d-4991-8351-70b9ad369db1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:32.151205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43836cdc-682d-4991-8351-70b9ad369db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:32.183006 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-43836cdc-682d-4991-8351-70b9ad369db1 None None] MFA Rules not processed for user `2c73c11b11c74540bb46302363604730`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:32.184653 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-43836cdc-682d-4991-8351-70b9ad369db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:32.226427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a52b3-de6b-4a9a-b5de-5fbd12f9ee79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:32.228031 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a52b3-de6b-4a9a-b5de-5fbd12f9ee79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:32.228359 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a52b3-de6b-4a9a-b5de-5fbd12f9ee79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:32.228647 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a52b3-de6b-4a9a-b5de-5fbd12f9ee79 admin admin] PATH_INFO: `/v3/projects/e870af07745948ffa62b2e59eb8054f4/users/a36fb9afb5d54d3493e8752bae88dd13/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:32.246110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a52b3-de6b-4a9a-b5de-5fbd12f9ee79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e870af07745948ffa62b2e59eb8054f4, user_id=a36fb9afb5d54d3493e8752bae88dd13, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:32.248999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a52b3-de6b-4a9a-b5de-5fbd12f9ee79 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:32.262956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 826/1608] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:32.262956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311cb81-715c-4124-b013-666d7c1ad404 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:32.262956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311cb81-715c-4124-b013-666d7c1ad404 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:32.262956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311cb81-715c-4124-b013-666d7c1ad404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:32.273158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 783/1609] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:32 2022] PUT /identity/v3/projects/e870af07745948ffa62b2e59eb8054f4/users/a36fb9afb5d54d3493e8752bae88dd13/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:32.275102 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf0db9-0d82-48c8-9368-3ac228d1ff67 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:32.275524 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf0db9-0d82-48c8-9368-3ac228d1ff67 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:32.276272 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf0db9-0d82-48c8-9368-3ac228d1ff67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:32.298553 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-e311cb81-715c-4124-b013-666d7c1ad404 None None] MFA Rules not processed for user `7ca6ef34e5624f868540d89c2a650e7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:32.300533 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e311cb81-715c-4124-b013-666d7c1ad404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:32.310855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a9cf0db9-0d82-48c8-9368-3ac228d1ff67 None None] MFA Rules not processed for user `ae50190c609c4e47ba9371410a07d867`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:32.312236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a9cf0db9-0d82-48c8-9368-3ac228d1ff67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:32.314244 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:32.406909 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 827/1610] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:32 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:32.406909 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311cb81-715c-4124-b013-666d7c1ad404 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:32.414788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 784/1611] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:32.414788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84be80e-92da-4ce1-89b2-9487f0e37fff None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:32.414788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84be80e-92da-4ce1-89b2-9487f0e37fff None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:32.414788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84be80e-92da-4ce1-89b2-9487f0e37fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:32.451475 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e84be80e-92da-4ce1-89b2-9487f0e37fff None None] MFA Rules not processed for user `2e14d892eae64aea8ef952d23cccdff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:32.452867 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e84be80e-92da-4ce1-89b2-9487f0e37fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:32.483041 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311cb81-715c-4124-b013-666d7c1ad404 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:32.487469 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7403720-6a92-4aa3-b415-bbd5dd615f8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:32.488144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7403720-6a92-4aa3-b415-bbd5dd615f8f service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:32.488570 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7403720-6a92-4aa3-b415-bbd5dd615f8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:32.489270 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7403720-6a92-4aa3-b415-bbd5dd615f8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:32.504525 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:32.538422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 785/1612] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:32.540737 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d202ae8-a68e-48d5-b075-88acf00dd108 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:32.541034 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d202ae8-a68e-48d5-b075-88acf00dd108 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:32.541372 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d202ae8-a68e-48d5-b075-88acf00dd108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:32.567762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1d202ae8-a68e-48d5-b075-88acf00dd108 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:32.569000 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1d202ae8-a68e-48d5-b075-88acf00dd108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:32.592495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7403720-6a92-4aa3-b415-bbd5dd615f8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:32.594368 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7403720-6a92-4aa3-b415-bbd5dd615f8f service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:32.607825 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 828/1613] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:32.609360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439c2f7-244c-4c23-9892-d6b3a2797e54 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:32.609656 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439c2f7-244c-4c23-9892-d6b3a2797e54 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:32.610080 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439c2f7-244c-4c23-9892-d6b3a2797e54 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:32.616718 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 829/1614] 10.222.0.67 () {54 vars in 961 bytes} [Tue May 31 14:38:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:38:32.617787 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439c2f7-244c-4c23-9892-d6b3a2797e54 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:32.631283 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439c2f7-244c-4c23-9892-d6b3a2797e54 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:32.636672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfc316-8ec1-4478-b487-221e82855474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:32.637488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfc316-8ec1-4478-b487-221e82855474 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:32.637786 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfc316-8ec1-4478-b487-221e82855474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:32.638082 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfc316-8ec1-4478-b487-221e82855474 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:32.638819 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfc316-8ec1-4478-b487-221e82855474 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:32.640885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfc316-8ec1-4478-b487-221e82855474 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:32.659106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 786/1615] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:32.661068 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8de16-a5a1-48b6-a9b7-dd77df848ce9 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:32.661765 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8de16-a5a1-48b6-a9b7-dd77df848ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:32.662147 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8de16-a5a1-48b6-a9b7-dd77df848ce9 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:32.664037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 787/1616] 10.222.0.67 () {54 vars in 961 bytes} [Tue May 31 14:38:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:38:32.665623 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8de16-a5a1-48b6-a9b7-dd77df848ce9 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:32.672825 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 830/1617] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:38:32 2022] POST /identity/v3/projects => generated 363 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:32.674382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2d913-ea03-49d0-b2e8-95f56ab6b2a9 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:32.674899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2d913-ea03-49d0-b2e8-95f56ab6b2a9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:32.675392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2d913-ea03-49d0-b2e8-95f56ab6b2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:32.686293 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8de16-a5a1-48b6-a9b7-dd77df848ce9 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:32.697073 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-0bc2d913-ea03-49d0-b2e8-95f56ab6b2a9 None None] MFA Rules not processed for user `68c6363b60954c3ebf50d2019f5f2fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:32.698769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0bc2d913-ea03-49d0-b2e8-95f56ab6b2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:32.739974 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:32.763900 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f85e83-db5a-4dd6-9b5d-adecd7ef5aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:32.764578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f85e83-db5a-4dd6-9b5d-adecd7ef5aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:32.764864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f85e83-db5a-4dd6-9b5d-adecd7ef5aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:32.765245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f85e83-db5a-4dd6-9b5d-adecd7ef5aa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:32.766065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f85e83-db5a-4dd6-9b5d-adecd7ef5aa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:32.768005 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f85e83-db5a-4dd6-9b5d-adecd7ef5aa2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:32.791439 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 831/1618] 10.222.0.67 () {58 vars in 1058 bytes} [Tue May 31 14:38:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:32.801106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf66d76-a32e-4c88-a771-c12796b6ab82 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:32.801581 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf66d76-a32e-4c88-a771-c12796b6ab82 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:32.801955 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf66d76-a32e-4c88-a771-c12796b6ab82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:32.821803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 788/1619] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:38:32 2022] POST /identity/v3/projects => generated 371 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:32.822659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-ecf66d76-a32e-4c88-a771-c12796b6ab82 None None] MFA Rules not processed for user `68c6363b60954c3ebf50d2019f5f2fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:32.823540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f85e83-db5a-4dd6-9b5d-adecd7ef5aa2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:32.823948 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ecf66d76-a32e-4c88-a771-c12796b6ab82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:32.911830 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f85e83-db5a-4dd6-9b5d-adecd7ef5aa2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:32.920435 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9203d8-2c32-4da0-9b79-835600f8eabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:32.921453 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9203d8-2c32-4da0-9b79-835600f8eabe admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:32.921907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9203d8-2c32-4da0-9b79-835600f8eabe admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:32.922394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9203d8-2c32-4da0-9b79-835600f8eabe admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:32.923293 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9203d8-2c32-4da0-9b79-835600f8eabe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:32.925952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9203d8-2c32-4da0-9b79-835600f8eabe admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:32.928274 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 832/1620] 10.222.0.67 () {58 vars in 1058 bytes} [Tue May 31 14:38:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:32.929253 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf66d76-a32e-4c88-a771-c12796b6ab82 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:32.949735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:32.958321 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 789/1621] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:38:32 2022] POST /identity/v3/users => generated 288 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:32.965004 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9203d8-2c32-4da0-9b79-835600f8eabe admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:32.984144 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9203d8-2c32-4da0-9b79-835600f8eabe admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:32.993031 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021e458-7eae-466e-987d-f2ff547b8f5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:32.994804 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021e458-7eae-466e-987d-f2ff547b8f5e service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:32.995464 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021e458-7eae-466e-987d-f2ff547b8f5e service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:32.995779 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021e458-7eae-466e-987d-f2ff547b8f5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:33.006744 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf66d76-a32e-4c88-a771-c12796b6ab82 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:33.011276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fdc43-d4ce-4327-9320-3c7c752bc645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:33.012010 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fdc43-d4ce-4327-9320-3c7c752bc645 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:33.012276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fdc43-d4ce-4327-9320-3c7c752bc645 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:33.012546 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fdc43-d4ce-4327-9320-3c7c752bc645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:33.025218 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7fdc43-d4ce-4327-9320-3c7c752bc645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:33.026558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7fdc43-d4ce-4327-9320-3c7c752bc645 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:33.038826 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 833/1622] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:33.040682 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fdc43-d4ce-4327-9320-3c7c752bc645 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:33.051361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fdc43-d4ce-4327-9320-3c7c752bc645 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:33.057566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a211b-7eeb-4d51-a5b3-dd6eae784793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:33.058255 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a211b-7eeb-4d51-a5b3-dd6eae784793 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:33.058589 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a211b-7eeb-4d51-a5b3-dd6eae784793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:33.058900 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a211b-7eeb-4d51-a5b3-dd6eae784793 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:33.060408 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a211b-7eeb-4d51-a5b3-dd6eae784793 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:33.062513 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a211b-7eeb-4d51-a5b3-dd6eae784793 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:33.087519 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 834/1623] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:38:33 2022] POST /identity/v3/users => generated 292 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:33.088367 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a211b-7eeb-4d51-a5b3-dd6eae784793 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:33.109711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a211b-7eeb-4d51-a5b3-dd6eae784793 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:33.123929 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251a15b-a485-4522-b233-02233c54ad44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:33.124666 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251a15b-a485-4522-b233-02233c54ad44 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:33.124964 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251a15b-a485-4522-b233-02233c54ad44 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:33.125300 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251a15b-a485-4522-b233-02233c54ad44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:33.142284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021e458-7eae-466e-987d-f2ff547b8f5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:33.143951 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021e458-7eae-466e-987d-f2ff547b8f5e service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:33.164601 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 790/1624] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:33.167184 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021e458-7eae-466e-987d-f2ff547b8f5e service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:33.190199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021e458-7eae-466e-987d-f2ff547b8f5e service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:33.204642 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76065d1-65c2-42dd-91c8-a5632fa56bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:33.205738 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76065d1-65c2-42dd-91c8-a5632fa56bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:33.206098 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76065d1-65c2-42dd-91c8-a5632fa56bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:33.206445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76065d1-65c2-42dd-91c8-a5632fa56bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:33.207906 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76065d1-65c2-42dd-91c8-a5632fa56bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:33.212647 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76065d1-65c2-42dd-91c8-a5632fa56bb5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:33.216137 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:33.236705 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3251a15b-a485-4522-b233-02233c54ad44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:33.238561 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 791/1625] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:33.239515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76065d1-65c2-42dd-91c8-a5632fa56bb5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:33.239892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3251a15b-a485-4522-b233-02233c54ad44 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:33.256109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 835/1626] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:33.259933 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251a15b-a485-4522-b233-02233c54ad44 service nova] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:33.265239 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76065d1-65c2-42dd-91c8-a5632fa56bb5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:33.275591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f89efa-5016-4cac-be49-021472ca9b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:33.276784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f89efa-5016-4cac-be49-021472ca9b0b admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:33.277070 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f89efa-5016-4cac-be49-021472ca9b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:33.277348 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f89efa-5016-4cac-be49-021472ca9b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:33.278441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f89efa-5016-4cac-be49-021472ca9b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:33.281248 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f89efa-5016-4cac-be49-021472ca9b0b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:33.290874 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251a15b-a485-4522-b233-02233c54ad44 service nova] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:33.304192 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ac86e-67f1-4eff-9b73-4cf879b58be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:33.307081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ac86e-67f1-4eff-9b73-4cf879b58be0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:33.307961 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ac86e-67f1-4eff-9b73-4cf879b58be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:33.308261 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ac86e-67f1-4eff-9b73-4cf879b58be0 admin admin] PATH_INFO: `/v3/projects/039c26e374e249689d9251848bb98ec0/users/f775c78c25584769b493376fe2d8fe0c/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:33.320918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 792/1627] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:33.325281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f89efa-5016-4cac-be49-021472ca9b0b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:33.357216 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ac86e-67f1-4eff-9b73-4cf879b58be0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=039c26e374e249689d9251848bb98ec0, user_id=f775c78c25584769b493376fe2d8fe0c, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:33.358197 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f89efa-5016-4cac-be49-021472ca9b0b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:33.360087 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ac86e-67f1-4eff-9b73-4cf879b58be0 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:33.369875 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b340800-fa0d-4a5f-b2b8-ed07f2cda170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:33.370975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b340800-fa0d-4a5f-b2b8-ed07f2cda170 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:33.371264 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b340800-fa0d-4a5f-b2b8-ed07f2cda170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:33.371541 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b340800-fa0d-4a5f-b2b8-ed07f2cda170 admin admin] PATH_INFO: `/v3/projects/98c76e5b4426488dbaf935e45fd3353e/users/b1f54ae9af6d4d949df5dfac318967b4/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:33.391876 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 836/1628] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:33 2022] PUT /identity/v3/projects/039c26e374e249689d9251848bb98ec0/users/f775c78c25584769b493376fe2d8fe0c/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:33.397334 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ac86e-67f1-4eff-9b73-4cf879b58be0 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:33.414530 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b340800-fa0d-4a5f-b2b8-ed07f2cda170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98c76e5b4426488dbaf935e45fd3353e, user_id=b1f54ae9af6d4d949df5dfac318967b4, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:33.423372 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b340800-fa0d-4a5f-b2b8-ed07f2cda170 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:33.456349 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 793/1629] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:33 2022] PUT /identity/v3/projects/98c76e5b4426488dbaf935e45fd3353e/users/b1f54ae9af6d4d949df5dfac318967b4/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:33.466500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b340800-fa0d-4a5f-b2b8-ed07f2cda170 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:33.600185 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ac86e-67f1-4eff-9b73-4cf879b58be0 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:33.644837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b340800-fa0d-4a5f-b2b8-ed07f2cda170 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:33.666779 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f034ef0-629d-4662-a7b5-0dff3eb69f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:33.669289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f034ef0-629d-4662-a7b5-0dff3eb69f95 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:33.669711 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f034ef0-629d-4662-a7b5-0dff3eb69f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:33.671204 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f034ef0-629d-4662-a7b5-0dff3eb69f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:33.674212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f034ef0-629d-4662-a7b5-0dff3eb69f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:33.676553 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f034ef0-629d-4662-a7b5-0dff3eb69f95 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:33.720703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 794/1630] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:33.722373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f034ef0-629d-4662-a7b5-0dff3eb69f95 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:33.750967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f034ef0-629d-4662-a7b5-0dff3eb69f95 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:33.760532 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a637fc-3630-4afd-8602-b2da675be4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:33.762866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a637fc-3630-4afd-8602-b2da675be4b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:33.764748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a637fc-3630-4afd-8602-b2da675be4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:33.765152 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a637fc-3630-4afd-8602-b2da675be4b0 admin admin] PATH_INFO: `/v3/projects/98c76e5b4426488dbaf935e45fd3353e/users/b1f54ae9af6d4d949df5dfac318967b4/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:33.769026 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a637fc-3630-4afd-8602-b2da675be4b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98c76e5b4426488dbaf935e45fd3353e, user_id=b1f54ae9af6d4d949df5dfac318967b4, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:33.775093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a637fc-3630-4afd-8602-b2da675be4b0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:33.786393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb584bd-8364-4253-ad4d-2da61634b3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:33.788251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb584bd-8364-4253-ad4d-2da61634b3af admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:33.788658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb584bd-8364-4253-ad4d-2da61634b3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:33.789786 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb584bd-8364-4253-ad4d-2da61634b3af admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:33.792339 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb584bd-8364-4253-ad4d-2da61634b3af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:33.795892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb584bd-8364-4253-ad4d-2da61634b3af admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:33.814199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 795/1631] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:33 2022] PUT /identity/v3/projects/98c76e5b4426488dbaf935e45fd3353e/users/b1f54ae9af6d4d949df5dfac318967b4/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:33.817344 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa97e6-b7bd-4fca-a9cc-c2d9d2279c87 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:33.817344 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa97e6-b7bd-4fca-a9cc-c2d9d2279c87 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:33.817344 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa97e6-b7bd-4fca-a9cc-c2d9d2279c87 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:33.818483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 837/1632] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:33.822064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a75589-225c-4720-9a63-7570a83e60f2 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:33.824087 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a75589-225c-4720-9a63-7570a83e60f2 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:33.826213 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 796/1633] 10.222.0.67 () {54 vars in 912 bytes} [Tue May 31 14:38:33 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:38:33.827207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa97e6-b7bd-4fca-a9cc-c2d9d2279c87 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:33.830568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a75589-225c-4720-9a63-7570a83e60f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:33.921624 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-24a75589-225c-4720-9a63-7570a83e60f2 None None] MFA Rules not processed for user `b1f54ae9af6d4d949df5dfac318967b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:33.926794 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-24a75589-225c-4720-9a63-7570a83e60f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:33.954701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:34.043836 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa97e6-b7bd-4fca-a9cc-c2d9d2279c87 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:34.048658 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65277212-b53f-4e46-af5c-be31496e6c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:34.049856 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65277212-b53f-4e46-af5c-be31496e6c48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:34.050258 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65277212-b53f-4e46-af5c-be31496e6c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:34.053870 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65277212-b53f-4e46-af5c-be31496e6c48 admin admin] PATH_INFO: `/v3/projects/039c26e374e249689d9251848bb98ec0/users/f775c78c25584769b493376fe2d8fe0c/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:34.058318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65277212-b53f-4e46-af5c-be31496e6c48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=039c26e374e249689d9251848bb98ec0, user_id=f775c78c25584769b493376fe2d8fe0c, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:34.062384 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65277212-b53f-4e46-af5c-be31496e6c48 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:34.071668 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:34.080581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 797/1634] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:33 2022] PUT /identity/v3/projects/039c26e374e249689d9251848bb98ec0/users/f775c78c25584769b493376fe2d8fe0c/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:34.084002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787dbad9-fa80-4353-979a-308d1d7569c6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:34.093139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787dbad9-fa80-4353-979a-308d1d7569c6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:34.093566 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787dbad9-fa80-4353-979a-308d1d7569c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:34.116043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 838/1635] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:33 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:34.118663 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de738a2-1712-405d-82f7-9dc84b8474d7 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:34.118977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de738a2-1712-405d-82f7-9dc84b8474d7 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:34.119259 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de738a2-1712-405d-82f7-9dc84b8474d7 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:34.121234 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 839/1636] 10.222.0.67 () {54 vars in 912 bytes} [Tue May 31 14:38:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:38:34.131223 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-787dbad9-fa80-4353-979a-308d1d7569c6 None None] MFA Rules not processed for user `68c6363b60954c3ebf50d2019f5f2fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:34.131336 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8692e-39d7-4ba1-ac28-b2fa7709e5f7 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:34.131336 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8692e-39d7-4ba1-ac28-b2fa7709e5f7 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:34.131336 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8692e-39d7-4ba1-ac28-b2fa7709e5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:34.132248 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-787dbad9-fa80-4353-979a-308d1d7569c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:34.176902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-84d8692e-39d7-4ba1-ac28-b2fa7709e5f7 None None] MFA Rules not processed for user `f775c78c25584769b493376fe2d8fe0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:34.178488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-84d8692e-39d7-4ba1-ac28-b2fa7709e5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:34.225045 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 798/1637] 10.222.0.67 () {58 vars in 1009 bytes} [Tue May 31 14:38:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:34.226559 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787dbad9-fa80-4353-979a-308d1d7569c6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:34.309143 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 840/1638] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:34 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:34.311460 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a052027-60e4-4686-90d5-8c49fdcd1fc9 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:34.312178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a052027-60e4-4686-90d5-8c49fdcd1fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:34.312996 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a052027-60e4-4686-90d5-8c49fdcd1fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:34.340400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787dbad9-fa80-4353-979a-308d1d7569c6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:34.345318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e039f78-4155-4e87-b440-614f72a75005 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:34.346395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e039f78-4155-4e87-b440-614f72a75005 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:34.346617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-8a052027-60e4-4686-90d5-8c49fdcd1fc9 None None] MFA Rules not processed for user `68c6363b60954c3ebf50d2019f5f2fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:34.347033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e039f78-4155-4e87-b440-614f72a75005 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:34.347653 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e039f78-4155-4e87-b440-614f72a75005 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:34.362516 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8a052027-60e4-4686-90d5-8c49fdcd1fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:34.392948 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 841/1639] 10.222.0.67 () {58 vars in 1009 bytes} [Tue May 31 14:38:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:34.394058 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a052027-60e4-4686-90d5-8c49fdcd1fc9 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:34.404930 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a052027-60e4-4686-90d5-8c49fdcd1fc9 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:34.408717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98a40a-5366-4354-8d72-47081b8e920c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:34.409557 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98a40a-5366-4354-8d72-47081b8e920c service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:34.410036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98a40a-5366-4354-8d72-47081b8e920c service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:34.410459 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98a40a-5366-4354-8d72-47081b8e920c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:34.423785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98a40a-5366-4354-8d72-47081b8e920c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:34.425465 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98a40a-5366-4354-8d72-47081b8e920c service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:34.460531 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 842/1640] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:34.467654 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e039f78-4155-4e87-b440-614f72a75005 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:34.469322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e039f78-4155-4e87-b440-614f72a75005 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:34.500505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 799/1641] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:34 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:34.506557 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98a40a-5366-4354-8d72-47081b8e920c service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:34.520938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98a40a-5366-4354-8d72-47081b8e920c service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:34.524823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d6708-3bfb-44b5-ad34-f37ac2cb79e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:34.527118 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d6708-3bfb-44b5-ad34-f37ac2cb79e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:34.527118 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d6708-3bfb-44b5-ad34-f37ac2cb79e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:34.527118 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d6708-3bfb-44b5-ad34-f37ac2cb79e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:34.541237 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0d6708-3bfb-44b5-ad34-f37ac2cb79e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:34.543193 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0d6708-3bfb-44b5-ad34-f37ac2cb79e5 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:34.571558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 843/1642] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:34 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:34.596216 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e039f78-4155-4e87-b440-614f72a75005 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:34.609137 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e039f78-4155-4e87-b440-614f72a75005 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:34.619222 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f2700-b5e5-474f-ae7b-768732b74dd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:34.620368 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f2700-b5e5-474f-ae7b-768732b74dd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:34.621310 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f2700-b5e5-474f-ae7b-768732b74dd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:34.622939 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f2700-b5e5-474f-ae7b-768732b74dd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:34.646911 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f2700-b5e5-474f-ae7b-768732b74dd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:34.652573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f2700-b5e5-474f-ae7b-768732b74dd4 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:34.668659 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:34.683683 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 800/1643] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:35.396316 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d6708-3bfb-44b5-ad34-f37ac2cb79e5 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:35.412891 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d6708-3bfb-44b5-ad34-f37ac2cb79e5 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:35.417761 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ca11d1-113f-4aa3-a360-fc6d7712b2c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:35.418418 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca11d1-113f-4aa3-a360-fc6d7712b2c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:35.418722 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca11d1-113f-4aa3-a360-fc6d7712b2c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:35.419002 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca11d1-113f-4aa3-a360-fc6d7712b2c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:35.540628 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f2700-b5e5-474f-ae7b-768732b74dd4 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:35.551444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ca11d1-113f-4aa3-a360-fc6d7712b2c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:35.552440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f2700-b5e5-474f-ae7b-768732b74dd4 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:35.553446 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ca11d1-113f-4aa3-a360-fc6d7712b2c1 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:35.562118 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feff4555-9d61-417d-9501-2f50d9c3477a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:35.562823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff4555-9d61-417d-9501-2f50d9c3477a service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:35.563191 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff4555-9d61-417d-9501-2f50d9c3477a service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:35.563537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff4555-9d61-417d-9501-2f50d9c3477a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:35.568655 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 844/1644] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:35.627470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab0b68-3278-4b79-ba0f-e1c7aff5068c None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:35.627747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab0b68-3278-4b79-ba0f-e1c7aff5068c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:35.628032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab0b68-3278-4b79-ba0f-e1c7aff5068c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:35.647570 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feff4555-9d61-417d-9501-2f50d9c3477a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:35.650812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feff4555-9d61-417d-9501-2f50d9c3477a service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:35.677430 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 801/1645] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:35 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:38:35.680385 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-f7ab0b68-3278-4b79-ba0f-e1c7aff5068c None None] MFA Rules not processed for user `ea126c816da740dda9622f7bfa10dd16`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:35.682106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f7ab0b68-3278-4b79-ba0f-e1c7aff5068c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:35.730866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:35.787864 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 845/1646] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:35.798702 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4ea48-db1c-44f4-ae0e-299befa80a44 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:35.799425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4ea48-db1c-44f4-ae0e-299befa80a44 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:35.800867 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4ea48-db1c-44f4-ae0e-299befa80a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:35.832313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-91b4ea48-db1c-44f4-ae0e-299befa80a44 None None] MFA Rules not processed for user `ea126c816da740dda9622f7bfa10dd16`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:35.833989 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-91b4ea48-db1c-44f4-ae0e-299befa80a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:35.887611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 802/1647] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:35.915723 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab0b68-3278-4b79-ba0f-e1c7aff5068c None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:35.947305 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab0b68-3278-4b79-ba0f-e1c7aff5068c None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:35.966151 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74520d2-ae89-497d-ba5e-c2dd686ff010 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:35.968481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74520d2-ae89-497d-ba5e-c2dd686ff010 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:35.968949 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74520d2-ae89-497d-ba5e-c2dd686ff010 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:35.969565 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74520d2-ae89-497d-ba5e-c2dd686ff010 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:35.979660 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74520d2-ae89-497d-ba5e-c2dd686ff010 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:35.982021 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74520d2-ae89-497d-ba5e-c2dd686ff010 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:36.006596 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4ea48-db1c-44f4-ae0e-299befa80a44 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:36.022341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 846/1648] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:36.168104 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4ea48-db1c-44f4-ae0e-299befa80a44 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:36.178977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e297bc-0e9d-4787-886d-4dab07a6bde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:36.179887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e297bc-0e9d-4787-886d-4dab07a6bde4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:36.180175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e297bc-0e9d-4787-886d-4dab07a6bde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:36.180987 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e297bc-0e9d-4787-886d-4dab07a6bde4 admin admin] PATH_INFO: `/v3/users/2c73c11b11c74540bb46302363604730` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:36.183859 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e297bc-0e9d-4787-886d-4dab07a6bde4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c73c11b11c74540bb46302363604730)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:36.184683 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-13e297bc-0e9d-4787-886d-4dab07a6bde4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 31 14:38:36.184683 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: warnings.warn(msg) May 31 14:38:36.184683 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: May 31 14:38:36.185893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e297bc-0e9d-4787-886d-4dab07a6bde4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:36.362703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-13e297bc-0e9d-4787-886d-4dab07a6bde4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c73c11b11c74540bb46302363604730'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:36.385660 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-13e297bc-0e9d-4787-886d-4dab07a6bde4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c73c11b11c74540bb46302363604730'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:36.416809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-13e297bc-0e9d-4787-886d-4dab07a6bde4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c73c11b11c74540bb46302363604730'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:36.451567 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-13e297bc-0e9d-4787-886d-4dab07a6bde4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c73c11b11c74540bb46302363604730'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:36.452482 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-13e297bc-0e9d-4787-886d-4dab07a6bde4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c73c11b11c74540bb46302363604730'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:36.469274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 803/1649] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:38:36 2022] DELETE /identity/v3/users/2c73c11b11c74540bb46302363604730 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:36.473903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4b60c-ef1d-4384-b888-e75682c05e6b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:36.474482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4b60c-ef1d-4384-b888-e75682c05e6b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:36.474572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4b60c-ef1d-4384-b888-e75682c05e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:36.503635 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6dd4b60c-ef1d-4384-b888-e75682c05e6b None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:36.504893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6dd4b60c-ef1d-4384-b888-e75682c05e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:36.647129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 847/1650] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:38:36.724721 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa8a1f-bd84-4d80-85e3-56ece8e11ce9 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:36.725182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa8a1f-bd84-4d80-85e3-56ece8e11ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:36.725341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa8a1f-bd84-4d80-85e3-56ece8e11ce9 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:36.726899 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 804/1651] 10.222.0.67 () {54 vars in 912 bytes} [Tue May 31 14:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:38:36.734264 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c118a-8b6d-4e30-988d-94e77498f34b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:36.734613 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c118a-8b6d-4e30-988d-94e77498f34b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:36.734922 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c118a-8b6d-4e30-988d-94e77498f34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:36.754437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa8a1f-bd84-4d80-85e3-56ece8e11ce9 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:36.756275 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-70aa8a1f-bd84-4d80-85e3-56ece8e11ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:36.777209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:36.846986 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-2a6c118a-8b6d-4e30-988d-94e77498f34b None None] MFA Rules not processed for user `8966bc2ea7ff497c84214c9c9e6d53a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:36.850907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2a6c118a-8b6d-4e30-988d-94e77498f34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:36.982038 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa8a1f-bd84-4d80-85e3-56ece8e11ce9 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:36.987455 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0a4ba-2f7f-49fa-b7f4-75aa41f2f1dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:36.988393 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0a4ba-2f7f-49fa-b7f4-75aa41f2f1dd service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:36.988781 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0a4ba-2f7f-49fa-b7f4-75aa41f2f1dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:36.989198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0a4ba-2f7f-49fa-b7f4-75aa41f2f1dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:37.012343 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd0a4ba-2f7f-49fa-b7f4-75aa41f2f1dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:37.016888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd0a4ba-2f7f-49fa-b7f4-75aa41f2f1dd service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:37.033904 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 848/1652] 10.222.0.67 () {58 vars in 1009 bytes} [Tue May 31 14:38:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:38:37.037653 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 805/1653] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:38:37.064508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c118a-8b6d-4e30-988d-94e77498f34b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:37.089116 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c118a-8b6d-4e30-988d-94e77498f34b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:37.099882 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03a355-9697-42e5-9423-f1b59e999a2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:37.100687 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03a355-9697-42e5-9423-f1b59e999a2e service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:37.108048 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03a355-9697-42e5-9423-f1b59e999a2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:37.108048 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03a355-9697-42e5-9423-f1b59e999a2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:37.133914 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03a355-9697-42e5-9423-f1b59e999a2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:37.136032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03a355-9697-42e5-9423-f1b59e999a2e service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:37.159954 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 849/1654] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:37.966956 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0a4ba-2f7f-49fa-b7f4-75aa41f2f1dd service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:37.969142 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7cd0a4ba-2f7f-49fa-b7f4-75aa41f2f1dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:38.009091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03a355-9697-42e5-9423-f1b59e999a2e service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:38.013065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1d03a355-9697-42e5-9423-f1b59e999a2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:38.046412 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03a355-9697-42e5-9423-f1b59e999a2e service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:38.047047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0a4ba-2f7f-49fa-b7f4-75aa41f2f1dd service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:38.073836 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f3071-9fe0-40db-87aa-b710d1469793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:38.074746 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f3071-9fe0-40db-87aa-b710d1469793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:38.075195 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f3071-9fe0-40db-87aa-b710d1469793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:38.075722 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f3071-9fe0-40db-87aa-b710d1469793 admin admin] PATH_INFO: `/v3/projects/2ebca4039f664206a5e793438f28b98f` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:38.077351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4f3071-9fe0-40db-87aa-b710d1469793 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ebca4039f664206a5e793438f28b98f)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:38.079009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: WARNING py.warnings [None req-1d4f3071-9fe0-40db-87aa-b710d1469793 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 31 14:38:38.079009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: warnings.warn(msg) May 31 14:38:38.079009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: May 31 14:38:38.082445 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4f3071-9fe0-40db-87aa-b710d1469793 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:38.086545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd906252-0921-43a7-99fe-b3025d892891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:38.087989 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd906252-0921-43a7-99fe-b3025d892891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:38.088634 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd906252-0921-43a7-99fe-b3025d892891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:38.089207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd906252-0921-43a7-99fe-b3025d892891 admin admin] PATH_INFO: `/v3/users/ae50190c609c4e47ba9371410a07d867` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:38.090891 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd906252-0921-43a7-99fe-b3025d892891 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae50190c609c4e47ba9371410a07d867)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:38.093461 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd906252-0921-43a7-99fe-b3025d892891 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:38.205383 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:38.241005 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1d4f3071-9fe0-40db-87aa-b710d1469793 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ebca4039f664206a5e793438f28b98f', 'request_id': 'req-1d4f3071-9fe0-40db-87aa-b710d1469793'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:38.257625 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1d4f3071-9fe0-40db-87aa-b710d1469793 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ebca4039f664206a5e793438f28b98f', 'request_id': 'req-1d4f3071-9fe0-40db-87aa-b710d1469793'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:38.272860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cd906252-0921-43a7-99fe-b3025d892891 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae50190c609c4e47ba9371410a07d867'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:38.292262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1d4f3071-9fe0-40db-87aa-b710d1469793 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ebca4039f664206a5e793438f28b98f', 'request_id': 'req-1d4f3071-9fe0-40db-87aa-b710d1469793'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:38.308553 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cd906252-0921-43a7-99fe-b3025d892891 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae50190c609c4e47ba9371410a07d867'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:38.318332 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1d4f3071-9fe0-40db-87aa-b710d1469793 admin admin] The token cache is being invalidate because project 2ebca4039f664206a5e793438f28b98f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:38:38.318596 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1d4f3071-9fe0-40db-87aa-b710d1469793 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:38.319232 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cd906252-0921-43a7-99fe-b3025d892891 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae50190c609c4e47ba9371410a07d867'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:38.319337 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1d4f3071-9fe0-40db-87aa-b710d1469793 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:38.323412 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 850/1655] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:38:38 2022] DELETE /identity/v3/projects/2ebca4039f664206a5e793438f28b98f => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:38.324876 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb78a8-edbb-4e4d-ad0a-6367d417d843 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:38.326426 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb78a8-edbb-4e4d-ad0a-6367d417d843 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:38.326771 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb78a8-edbb-4e4d-ad0a-6367d417d843 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:38.331541 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 851/1656] 10.222.0.67 () {54 vars in 960 bytes} [Tue May 31 14:38:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:38:38.332803 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b97aa-5d43-43d5-92d0-bae025f486f8 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:38.333095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b97aa-5d43-43d5-92d0-bae025f486f8 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:38.333377 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b97aa-5d43-43d5-92d0-bae025f486f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:38.345782 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cd906252-0921-43a7-99fe-b3025d892891 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae50190c609c4e47ba9371410a07d867'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:38.346914 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cd906252-0921-43a7-99fe-b3025d892891 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae50190c609c4e47ba9371410a07d867'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:38.360671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 806/1657] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:38:37 2022] DELETE /identity/v3/users/ae50190c609c4e47ba9371410a07d867 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:38.363360 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25e0ce-0f54-468b-aa28-6f6096f66b98 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:38.364368 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25e0ce-0f54-468b-aa28-6f6096f66b98 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:38.364816 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25e0ce-0f54-468b-aa28-6f6096f66b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:38.383490 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a98b97aa-5d43-43d5-92d0-bae025f486f8 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:38.385062 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a98b97aa-5d43-43d5-92d0-bae025f486f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:38.390809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0e25e0ce-0f54-468b-aa28-6f6096f66b98 None None] MFA Rules not processed for user `6c1c13515a8b41a781478dc65e6a88b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:38.393282 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0e25e0ce-0f54-468b-aa28-6f6096f66b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:38.526885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 852/1658] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:38.528667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9187906-e9cc-4dd2-b3cd-f991380f933b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:38.529313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9187906-e9cc-4dd2-b3cd-f991380f933b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:38.529778 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9187906-e9cc-4dd2-b3cd-f991380f933b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:38.565953 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a9187906-e9cc-4dd2-b3cd-f991380f933b None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:38.567461 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a9187906-e9cc-4dd2-b3cd-f991380f933b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:38.567760 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 807/1659] 10.222.0.67 () {58 vars in 1057 bytes} [Tue May 31 14:38:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:38.577980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecf246-2e17-4e44-bac8-9985dd5eceb1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:38.578577 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecf246-2e17-4e44-bac8-9985dd5eceb1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:38.579572 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecf246-2e17-4e44-bac8-9985dd5eceb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:38.580367 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 853/1660] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:38.585658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9187906-e9cc-4dd2-b3cd-f991380f933b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:38.602597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9187906-e9cc-4dd2-b3cd-f991380f933b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:38.608748 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9946848-1ead-4746-9bb9-f788967c18cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c1c13515a8b41a781478dc65e6a88b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:38.609313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9946848-1ead-4746-9bb9-f788967c18cc service placement] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:38.609698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9946848-1ead-4746-9bb9-f788967c18cc service placement] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:38.610086 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9946848-1ead-4746-9bb9-f788967c18cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:38.612390 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a9946848-1ead-4746-9bb9-f788967c18cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:38.628227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9946848-1ead-4746-9bb9-f788967c18cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:38.631758 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c5ecf246-2e17-4e44-bac8-9985dd5eceb1 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:38.635686 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c5ecf246-2e17-4e44-bac8-9985dd5eceb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:38.639812 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9946848-1ead-4746-9bb9-f788967c18cc service placement] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:38.658059 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 808/1661] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:38.658059 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ecf246-2e17-4e44-bac8-9985dd5eceb1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:38.661584 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 854/1662] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:38.662564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9946848-1ead-4746-9bb9-f788967c18cc service placement] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:38.678976 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c5ecf246-2e17-4e44-bac8-9985dd5eceb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:38.679175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9946848-1ead-4746-9bb9-f788967c18cc service placement] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:38.687671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b8af3-9ede-4ca4-aac2-5fb9678de9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:38.688973 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b8af3-9ede-4ca4-aac2-5fb9678de9de admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:38.689398 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b8af3-9ede-4ca4-aac2-5fb9678de9de admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:38.689778 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b8af3-9ede-4ca4-aac2-5fb9678de9de admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:38.691024 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b8af3-9ede-4ca4-aac2-5fb9678de9de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:38:38.691436 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b8af3-9ede-4ca4-aac2-5fb9678de9de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:38.701505 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b8af3-9ede-4ca4-aac2-5fb9678de9de admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:38.722043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 855/1663] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:38:38.723165 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b8af3-9ede-4ca4-aac2-5fb9678de9de admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:38.730764 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:38.748760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b8af3-9ede-4ca4-aac2-5fb9678de9de admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:38.757561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d758902-34ec-4e80-b3a3-7ad30a9331c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:38.758416 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d758902-34ec-4e80-b3a3-7ad30a9331c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:38.762394 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d758902-34ec-4e80-b3a3-7ad30a9331c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:38.763342 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d758902-34ec-4e80-b3a3-7ad30a9331c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:38.765291 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d758902-34ec-4e80-b3a3-7ad30a9331c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:38.769294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d758902-34ec-4e80-b3a3-7ad30a9331c1 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:38.858111 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ecf246-2e17-4e44-bac8-9985dd5eceb1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:38.861765 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 856/1664] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:38:38 2022] POST /identity/v3/projects => generated 363 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:38.869884 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d758902-34ec-4e80-b3a3-7ad30a9331c1 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:39.003377 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5eb2d-cea7-4f81-8a87-4271c4f3112e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:39.004214 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5eb2d-cea7-4f81-8a87-4271c4f3112e service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:39.005808 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5eb2d-cea7-4f81-8a87-4271c4f3112e service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:39.006120 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5eb2d-cea7-4f81-8a87-4271c4f3112e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:39.095741 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d758902-34ec-4e80-b3a3-7ad30a9331c1 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:39.116404 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcaa13f-ba24-4f44-ba48-ccee9e38b7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:39.121793 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcaa13f-ba24-4f44-ba48-ccee9e38b7eb admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:39.121793 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcaa13f-ba24-4f44-ba48-ccee9e38b7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:39.121793 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcaa13f-ba24-4f44-ba48-ccee9e38b7eb admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:39.121793 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcaa13f-ba24-4f44-ba48-ccee9e38b7eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:39.129089 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcaa13f-ba24-4f44-ba48-ccee9e38b7eb admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:39.151594 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:39.167538 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 857/1665] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:38:38 2022] POST /identity/v3/users => generated 282 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:39.168509 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcaa13f-ba24-4f44-ba48-ccee9e38b7eb admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:39.194279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcaa13f-ba24-4f44-ba48-ccee9e38b7eb admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:39.208182 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb54ded-6854-4a73-b7e6-bec4a8cc397b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:39.209238 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb54ded-6854-4a73-b7e6-bec4a8cc397b admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:39.209238 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb54ded-6854-4a73-b7e6-bec4a8cc397b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:39.209445 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb54ded-6854-4a73-b7e6-bec4a8cc397b admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:39.210162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb54ded-6854-4a73-b7e6-bec4a8cc397b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:39.211876 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb54ded-6854-4a73-b7e6-bec4a8cc397b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:39.224420 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5eb2d-cea7-4f81-8a87-4271c4f3112e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:39.226676 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5eb2d-cea7-4f81-8a87-4271c4f3112e service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:39.250437 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 858/1666] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:39.254571 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb54ded-6854-4a73-b7e6-bec4a8cc397b admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:39.270404 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb54ded-6854-4a73-b7e6-bec4a8cc397b admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:39.287969 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 809/1667] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:39.293566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990fb444-e263-4ebe-a800-a62c30c7a059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:39.294423 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fb444-e263-4ebe-a800-a62c30c7a059 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:39.294772 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fb444-e263-4ebe-a800-a62c30c7a059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:39.295061 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fb444-e263-4ebe-a800-a62c30c7a059 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:39.295808 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990fb444-e263-4ebe-a800-a62c30c7a059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:39.297604 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990fb444-e263-4ebe-a800-a62c30c7a059 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:39.328234 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 859/1668] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:39.329881 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5eb2d-cea7-4f81-8a87-4271c4f3112e service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:39.349504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5eb2d-cea7-4f81-8a87-4271c4f3112e service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:39.372259 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06850e2-3335-492d-a61f-d80c8b9bccdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:39.373209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06850e2-3335-492d-a61f-d80c8b9bccdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:39.377746 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06850e2-3335-492d-a61f-d80c8b9bccdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:39.378105 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06850e2-3335-492d-a61f-d80c8b9bccdd admin admin] PATH_INFO: `/v3/projects/3263e2a380ae428bb802bad8856e2ba9/users/097a49d6e8ae4ebaba127e9fc585df2f/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:39.438226 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06850e2-3335-492d-a61f-d80c8b9bccdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3263e2a380ae428bb802bad8856e2ba9, user_id=097a49d6e8ae4ebaba127e9fc585df2f, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:39.445300 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06850e2-3335-492d-a61f-d80c8b9bccdd admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:39.491292 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 810/1669] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:39 2022] PUT /identity/v3/projects/3263e2a380ae428bb802bad8856e2ba9/users/097a49d6e8ae4ebaba127e9fc585df2f/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:39.503683 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650b9f4-4bc5-40aa-b6f2-fa1e99bd5e11 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:39.503972 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650b9f4-4bc5-40aa-b6f2-fa1e99bd5e11 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:39.504305 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650b9f4-4bc5-40aa-b6f2-fa1e99bd5e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:39.569024 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5650b9f4-4bc5-40aa-b6f2-fa1e99bd5e11 None None] MFA Rules not processed for user `097a49d6e8ae4ebaba127e9fc585df2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:39.572394 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5650b9f4-4bc5-40aa-b6f2-fa1e99bd5e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:39.776577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 860/1670] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:39 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:39.780748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06850e2-3335-492d-a61f-d80c8b9bccdd admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:39.947177 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:39.964241 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06850e2-3335-492d-a61f-d80c8b9bccdd admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:39.978789 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a52e3-77d7-4980-adb8-53e8c28883a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:39.979500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a52e3-77d7-4980-adb8-53e8c28883a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:39.979802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a52e3-77d7-4980-adb8-53e8c28883a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:39.980112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a52e3-77d7-4980-adb8-53e8c28883a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:39.980871 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a52e3-77d7-4980-adb8-53e8c28883a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:39.982915 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a52e3-77d7-4980-adb8-53e8c28883a4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:40.038754 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 811/1671] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:38:39 2022] POST /identity/v3/projects => generated 363 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:40.042070 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650b9f4-4bc5-40aa-b6f2-fa1e99bd5e11 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:40.059517 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650b9f4-4bc5-40aa-b6f2-fa1e99bd5e11 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:40.090141 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a52e3-77d7-4980-adb8-53e8c28883a4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:40.092510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-133a52e3-77d7-4980-adb8-53e8c28883a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:40.268067 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cf5333-8be4-4094-9dd4-109f5411ce17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:40.270621 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cf5333-8be4-4094-9dd4-109f5411ce17 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:40.271007 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cf5333-8be4-4094-9dd4-109f5411ce17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:40.271561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cf5333-8be4-4094-9dd4-109f5411ce17 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:40.273972 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cf5333-8be4-4094-9dd4-109f5411ce17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:40.278169 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cf5333-8be4-4094-9dd4-109f5411ce17 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:40.295581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a52e3-77d7-4980-adb8-53e8c28883a4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:40.310659 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf4d73-8306-4e1e-998e-e5a9d6d06172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:40.311873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf4d73-8306-4e1e-998e-e5a9d6d06172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:40.312247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf4d73-8306-4e1e-998e-e5a9d6d06172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:40.312594 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf4d73-8306-4e1e-998e-e5a9d6d06172 admin admin] PATH_INFO: `/v3/projects/37b130e093da409daa4f34baccfc96cd` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:40.321434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf4d73-8306-4e1e-998e-e5a9d6d06172 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37b130e093da409daa4f34baccfc96cd)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:40.323553 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf4d73-8306-4e1e-998e-e5a9d6d06172 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:40.338506 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 861/1672] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:38:40 2022] POST /identity/v3/users => generated 288 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:40.344024 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cf5333-8be4-4094-9dd4-109f5411ce17 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:40.392448 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8ddf4d73-8306-4e1e-998e-e5a9d6d06172 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37b130e093da409daa4f34baccfc96cd', 'request_id': 'req-8ddf4d73-8306-4e1e-998e-e5a9d6d06172'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:40.394558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cf5333-8be4-4094-9dd4-109f5411ce17 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:40.400846 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8ddf4d73-8306-4e1e-998e-e5a9d6d06172 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37b130e093da409daa4f34baccfc96cd', 'request_id': 'req-8ddf4d73-8306-4e1e-998e-e5a9d6d06172'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:40.408916 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8ddf4d73-8306-4e1e-998e-e5a9d6d06172 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37b130e093da409daa4f34baccfc96cd', 'request_id': 'req-8ddf4d73-8306-4e1e-998e-e5a9d6d06172'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:40.443542 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8ddf4d73-8306-4e1e-998e-e5a9d6d06172 admin admin] The token cache is being invalidate because project 37b130e093da409daa4f34baccfc96cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:38:40.443859 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8ddf4d73-8306-4e1e-998e-e5a9d6d06172 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:40.450019 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8ddf4d73-8306-4e1e-998e-e5a9d6d06172 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:40.456178 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 812/1673] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:38:40 2022] DELETE /identity/v3/projects/37b130e093da409daa4f34baccfc96cd => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:38:40.458049 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d82b4-5d24-4a2a-ae9f-f11c03786b25 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:40.458353 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d82b4-5d24-4a2a-ae9f-f11c03786b25 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:40.459485 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d82b4-5d24-4a2a-ae9f-f11c03786b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:40.481820 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-192d82b4-5d24-4a2a-ae9f-f11c03786b25 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:40.483111 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-192d82b4-5d24-4a2a-ae9f-f11c03786b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:40.537707 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:40.554315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2cd7b-6802-4e9e-a83d-8af533b6e295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:40.555071 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2cd7b-6802-4e9e-a83d-8af533b6e295 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:40.556090 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2cd7b-6802-4e9e-a83d-8af533b6e295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:40.556395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2cd7b-6802-4e9e-a83d-8af533b6e295 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:40.557171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2cd7b-6802-4e9e-a83d-8af533b6e295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:40.558833 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2cd7b-6802-4e9e-a83d-8af533b6e295 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:40.574413 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 813/1674] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:40.582675 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789acfb-9af6-4f33-8955-05362909b125 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:40.585292 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789acfb-9af6-4f33-8955-05362909b125 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:40.585583 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789acfb-9af6-4f33-8955-05362909b125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:40.585820 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 862/1675] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:40.590704 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2cd7b-6802-4e9e-a83d-8af533b6e295 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:40.604088 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e4d2cd7b-6802-4e9e-a83d-8af533b6e295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:40.609679 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1789acfb-9af6-4f33-8955-05362909b125 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:40.611135 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1789acfb-9af6-4f33-8955-05362909b125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:40.625195 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2cd7b-6802-4e9e-a83d-8af533b6e295 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:40.634888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 814/1676] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:40.636013 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789acfb-9af6-4f33-8955-05362909b125 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:40.638583 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d92694-3785-494b-af79-3716e890563c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:40.639611 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d92694-3785-494b-af79-3716e890563c admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:40.639956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d92694-3785-494b-af79-3716e890563c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:40.640314 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d92694-3785-494b-af79-3716e890563c admin admin] PATH_INFO: `/v3/projects/b34f1f2dc22a4dea9dd1129c234ba55c/users/508c3f83bddf4c38b3c09ffbe6f41214/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:40.644613 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:40.699390 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789acfb-9af6-4f33-8955-05362909b125 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:40.709960 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf0938-9297-4582-b647-6a14cc43e64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:40.711217 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf0938-9297-4582-b647-6a14cc43e64b admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:40.712054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf0938-9297-4582-b647-6a14cc43e64b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:40.712239 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d92694-3785-494b-af79-3716e890563c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b34f1f2dc22a4dea9dd1129c234ba55c, user_id=508c3f83bddf4c38b3c09ffbe6f41214, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:40.712345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf0938-9297-4582-b647-6a14cc43e64b admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:40.713068 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abf0938-9297-4582-b647-6a14cc43e64b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:38:40.713379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abf0938-9297-4582-b647-6a14cc43e64b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:40.714954 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d92694-3785-494b-af79-3716e890563c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:40.719611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abf0938-9297-4582-b647-6a14cc43e64b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:40.740109 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 815/1677] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:38:40.755340 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf0938-9297-4582-b647-6a14cc43e64b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:40.766536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf0938-9297-4582-b647-6a14cc43e64b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:40.771472 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 863/1678] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:40 2022] PUT /identity/v3/projects/b34f1f2dc22a4dea9dd1129c234ba55c/users/508c3f83bddf4c38b3c09ffbe6f41214/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:40.776281 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d92694-3785-494b-af79-3716e890563c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:40.920019 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ca68f-5b21-4506-ab40-1c4c9afad4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:40.920793 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ca68f-5b21-4506-ab40-1c4c9afad4c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:40.921296 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ca68f-5b21-4506-ab40-1c4c9afad4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:40.921651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ca68f-5b21-4506-ab40-1c4c9afad4c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:40.922426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ca68f-5b21-4506-ab40-1c4c9afad4c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:40.928834 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ca68f-5b21-4506-ab40-1c4c9afad4c0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:41.007851 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d92694-3785-494b-af79-3716e890563c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:41.026752 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 816/1679] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:38:40 2022] POST /identity/v3/projects => generated 373 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:41.027616 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906776e-917c-4c03-bc87-ec9db0218900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:41.028142 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ca68f-5b21-4506-ab40-1c4c9afad4c0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:41.028314 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906776e-917c-4c03-bc87-ec9db0218900 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:41.028606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906776e-917c-4c03-bc87-ec9db0218900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:41.028885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906776e-917c-4c03-bc87-ec9db0218900 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:41.029634 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906776e-917c-4c03-bc87-ec9db0218900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:41.037607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906776e-917c-4c03-bc87-ec9db0218900 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:41.064984 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 864/1680] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:41.070600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906776e-917c-4c03-bc87-ec9db0218900 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:41.105215 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906776e-917c-4c03-bc87-ec9db0218900 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:41.191139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ca68f-5b21-4506-ab40-1c4c9afad4c0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:41.201870 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cd8d3-4485-442f-a6fa-22c1f7b7b75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:41.202719 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cd8d3-4485-442f-a6fa-22c1f7b7b75a admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:41.203095 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cd8d3-4485-442f-a6fa-22c1f7b7b75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:41.203434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cd8d3-4485-442f-a6fa-22c1f7b7b75a admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:41.205918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3cd8d3-4485-442f-a6fa-22c1f7b7b75a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:41.214936 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3cd8d3-4485-442f-a6fa-22c1f7b7b75a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:41.273009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 817/1681] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:38:41 2022] POST /identity/v3/users => generated 287 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:41.274078 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cd8d3-4485-442f-a6fa-22c1f7b7b75a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:41.277960 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39773d1a-de21-4cc9-9d10-ef561692672a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:41.280191 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39773d1a-de21-4cc9-9d10-ef561692672a admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:41.280784 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39773d1a-de21-4cc9-9d10-ef561692672a admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:41.281243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39773d1a-de21-4cc9-9d10-ef561692672a admin admin] PATH_INFO: `/v3/projects/b34f1f2dc22a4dea9dd1129c234ba55c/users/508c3f83bddf4c38b3c09ffbe6f41214/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:41.284156 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39773d1a-de21-4cc9-9d10-ef561692672a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b34f1f2dc22a4dea9dd1129c234ba55c, user_id=508c3f83bddf4c38b3c09ffbe6f41214, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:41.287497 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39773d1a-de21-4cc9-9d10-ef561692672a admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:41.299423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cd8d3-4485-442f-a6fa-22c1f7b7b75a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:41.322125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5562e3-c0fb-4edb-96c5-2a6d659895fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:41.322735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5562e3-c0fb-4edb-96c5-2a6d659895fc admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:41.323175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5562e3-c0fb-4edb-96c5-2a6d659895fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:41.323540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5562e3-c0fb-4edb-96c5-2a6d659895fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:41.324202 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5562e3-c0fb-4edb-96c5-2a6d659895fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:41.325840 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5562e3-c0fb-4edb-96c5-2a6d659895fc admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:41.339427 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 865/1682] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:41 2022] PUT /identity/v3/projects/b34f1f2dc22a4dea9dd1129c234ba55c/users/508c3f83bddf4c38b3c09ffbe6f41214/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:41.352698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23cac9-3211-4c00-9295-0c4bd212f717 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:41.353635 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23cac9-3211-4c00-9295-0c4bd212f717 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:41.353906 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23cac9-3211-4c00-9295-0c4bd212f717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:41.377703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 818/1683] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:41.378842 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5562e3-c0fb-4edb-96c5-2a6d659895fc admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:41.393931 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5562e3-c0fb-4edb-96c5-2a6d659895fc admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:41.413424 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5c23cac9-3211-4c00-9295-0c4bd212f717 None None] MFA Rules not processed for user `508c3f83bddf4c38b3c09ffbe6f41214`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:41.414826 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5c23cac9-3211-4c00-9295-0c4bd212f717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:41.502562 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:41.506731 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:41.516119 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbaa82-893c-4135-9a8e-90731d8c98c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:41.516824 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbaa82-893c-4135-9a8e-90731d8c98c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:41.517108 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbaa82-893c-4135-9a8e-90731d8c98c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:41.517392 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbaa82-893c-4135-9a8e-90731d8c98c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:41.518241 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dbaa82-893c-4135-9a8e-90731d8c98c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:41.524963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dbaa82-893c-4135-9a8e-90731d8c98c5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:41.546097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 819/1684] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:38:41.551250 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbaa82-893c-4135-9a8e-90731d8c98c5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:41.581153 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 866/1685] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:41 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:41.585954 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbaa82-893c-4135-9a8e-90731d8c98c5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:41.596282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23cac9-3211-4c00-9295-0c4bd212f717 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:41.598268 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5c23cac9-3211-4c00-9295-0c4bd212f717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:41.599427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7acc683-36e1-47e6-92de-150080562fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:41.600343 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acc683-36e1-47e6-92de-150080562fe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:41.600631 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acc683-36e1-47e6-92de-150080562fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:41.600912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acc683-36e1-47e6-92de-150080562fe2 admin admin] PATH_INFO: `/v3/projects/95da8209d46f4d8683befcd8b59305d2/users/6052d52bd3cd4b009b51987e12229247/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:41.636301 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7acc683-36e1-47e6-92de-150080562fe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95da8209d46f4d8683befcd8b59305d2, user_id=6052d52bd3cd4b009b51987e12229247, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:41.639039 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7acc683-36e1-47e6-92de-150080562fe2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:41.680759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 820/1686] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:41 2022] PUT /identity/v3/projects/95da8209d46f4d8683befcd8b59305d2/users/6052d52bd3cd4b009b51987e12229247/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:38:41.685297 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c6900-dc45-4e17-a93e-5b72dbfbe81a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:41.685564 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c6900-dc45-4e17-a93e-5b72dbfbe81a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:41.685904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c6900-dc45-4e17-a93e-5b72dbfbe81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:41.745563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23cac9-3211-4c00-9295-0c4bd212f717 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:41.747659 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e15c6900-dc45-4e17-a93e-5b72dbfbe81a None None] MFA Rules not processed for user `6052d52bd3cd4b009b51987e12229247`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:41.749206 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e15c6900-dc45-4e17-a93e-5b72dbfbe81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:41.859642 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43d40c-77fa-49a8-999b-e9b10d1f2f0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:41.860321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43d40c-77fa-49a8-999b-e9b10d1f2f0f service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:41.860607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43d40c-77fa-49a8-999b-e9b10d1f2f0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:41.860903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43d40c-77fa-49a8-999b-e9b10d1f2f0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:41.899461 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 821/1687] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:38:41.911648 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c6900-dc45-4e17-a93e-5b72dbfbe81a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:41.980411 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b43d40c-77fa-49a8-999b-e9b10d1f2f0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:41.982418 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b43d40c-77fa-49a8-999b-e9b10d1f2f0f service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:41.994297 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 867/1688] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:41 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:42.045640 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c6900-dc45-4e17-a93e-5b72dbfbe81a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:42.050834 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70c0a7-717d-4b15-9602-57584a97cb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:42.051758 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70c0a7-717d-4b15-9602-57584a97cb22 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:42.052236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70c0a7-717d-4b15-9602-57584a97cb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:42.052681 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70c0a7-717d-4b15-9602-57584a97cb22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:42.053571 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d70c0a7-717d-4b15-9602-57584a97cb22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:42.055772 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d70c0a7-717d-4b15-9602-57584a97cb22 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:42.102921 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 822/1689] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:38:41 2022] POST /identity/v3/projects => generated 371 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:42.107128 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43d40c-77fa-49a8-999b-e9b10d1f2f0f service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:42.119903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43d40c-77fa-49a8-999b-e9b10d1f2f0f service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:42.234018 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:42.248737 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd425d-37b9-4d17-9cf0-77435fc54e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:42.249569 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd425d-37b9-4d17-9cf0-77435fc54e2c admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:42.249857 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd425d-37b9-4d17-9cf0-77435fc54e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:42.250300 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd425d-37b9-4d17-9cf0-77435fc54e2c admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:42.251053 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd425d-37b9-4d17-9cf0-77435fc54e2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:42.253131 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd425d-37b9-4d17-9cf0-77435fc54e2c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:42.308930 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 868/1690] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:38:42 2022] POST /identity/v3/users => generated 292 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:42.313749 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70c0a7-717d-4b15-9602-57584a97cb22 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:42.351107 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70c0a7-717d-4b15-9602-57584a97cb22 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:42.383857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f3a97-004c-4967-bf11-aba8d40e941f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:42.384517 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f3a97-004c-4967-bf11-aba8d40e941f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:42.384807 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f3a97-004c-4967-bf11-aba8d40e941f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:42.385090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f3a97-004c-4967-bf11-aba8d40e941f admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:42.386516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0f3a97-004c-4967-bf11-aba8d40e941f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:42.388611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0f3a97-004c-4967-bf11-aba8d40e941f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:42.426588 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd425d-37b9-4d17-9cf0-77435fc54e2c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:42.427428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 823/1691] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:42.442709 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd425d-37b9-4d17-9cf0-77435fc54e2c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:42.449010 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417d6a6-c566-4dea-8941-c5da41407c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:42.450491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417d6a6-c566-4dea-8941-c5da41407c91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:42.451039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417d6a6-c566-4dea-8941-c5da41407c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:42.455863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417d6a6-c566-4dea-8941-c5da41407c91 admin admin] PATH_INFO: `/v3/projects/656565f2f9c64cbf9c3c33388ffb779e/users/35cbfe2de26143e18e6bc10ffdc6cc55/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:42.502765 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417d6a6-c566-4dea-8941-c5da41407c91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=656565f2f9c64cbf9c3c33388ffb779e, user_id=35cbfe2de26143e18e6bc10ffdc6cc55, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:42.505500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417d6a6-c566-4dea-8941-c5da41407c91 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:42.551993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 869/1692] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:42 2022] PUT /identity/v3/projects/656565f2f9c64cbf9c3c33388ffb779e/users/35cbfe2de26143e18e6bc10ffdc6cc55/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:42.556332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f3a97-004c-4967-bf11-aba8d40e941f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:42.578063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f3a97-004c-4967-bf11-aba8d40e941f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:42.677069 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:42.748951 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807de15-ea0a-4ddb-8970-7ba60164ea56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:42.749588 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807de15-ea0a-4ddb-8970-7ba60164ea56 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:42.749999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807de15-ea0a-4ddb-8970-7ba60164ea56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:42.750315 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807de15-ea0a-4ddb-8970-7ba60164ea56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:42.751067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3807de15-ea0a-4ddb-8970-7ba60164ea56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:42.755715 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3807de15-ea0a-4ddb-8970-7ba60164ea56 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:42.790086 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 824/1693] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:42.794800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417d6a6-c566-4dea-8941-c5da41407c91 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:42.817558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417d6a6-c566-4dea-8941-c5da41407c91 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:42.838861 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84678f8-09ca-43aa-a01a-c67c0d7d96d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:42.839784 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84678f8-09ca-43aa-a01a-c67c0d7d96d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:42.841644 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84678f8-09ca-43aa-a01a-c67c0d7d96d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:42.843120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84678f8-09ca-43aa-a01a-c67c0d7d96d5 admin admin] PATH_INFO: `/v3/projects/656565f2f9c64cbf9c3c33388ffb779e/users/35cbfe2de26143e18e6bc10ffdc6cc55/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:42.848428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84678f8-09ca-43aa-a01a-c67c0d7d96d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=656565f2f9c64cbf9c3c33388ffb779e, user_id=35cbfe2de26143e18e6bc10ffdc6cc55, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:42.865971 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84678f8-09ca-43aa-a01a-c67c0d7d96d5 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:42.880726 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807de15-ea0a-4ddb-8970-7ba60164ea56 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:42.890148 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 870/1694] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:42 2022] PUT /identity/v3/projects/656565f2f9c64cbf9c3c33388ffb779e/users/35cbfe2de26143e18e6bc10ffdc6cc55/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:42.903473 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e0caba-7dea-4714-80d6-83fb8317d154 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:42.904005 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e0caba-7dea-4714-80d6-83fb8317d154 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:42.904651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e0caba-7dea-4714-80d6-83fb8317d154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:42.906200 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807de15-ea0a-4ddb-8970-7ba60164ea56 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:42.991055 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a7e0caba-7dea-4714-80d6-83fb8317d154 None None] MFA Rules not processed for user `35cbfe2de26143e18e6bc10ffdc6cc55`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:42.992327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a7e0caba-7dea-4714-80d6-83fb8317d154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:43.062193 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dfa278-e57b-4bdf-8196-4e48a3e06457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:43.065535 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dfa278-e57b-4bdf-8196-4e48a3e06457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:43.065833 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dfa278-e57b-4bdf-8196-4e48a3e06457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:43.066411 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dfa278-e57b-4bdf-8196-4e48a3e06457 admin admin] PATH_INFO: `/v3/users/097a49d6e8ae4ebaba127e9fc585df2f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:43.068962 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dfa278-e57b-4bdf-8196-4e48a3e06457 admin admin] RBAC: Authorizing `identity:delete_user(user_id=097a49d6e8ae4ebaba127e9fc585df2f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:43.072721 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dfa278-e57b-4bdf-8196-4e48a3e06457 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:43.132019 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 871/1695] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:42 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:43.136473 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5b707-f90d-4d05-8e54-a5173c1e4d1b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:43.138523 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5b707-f90d-4d05-8e54-a5173c1e4d1b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:43.139353 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5b707-f90d-4d05-8e54-a5173c1e4d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:43.184416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-02dfa278-e57b-4bdf-8196-4e48a3e06457 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '097a49d6e8ae4ebaba127e9fc585df2f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:43.194756 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-49b5b707-f90d-4d05-8e54-a5173c1e4d1b None None] MFA Rules not processed for user `91255c0b2d914ef9be10eb69d7d8ed4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:43.197090 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-49b5b707-f90d-4d05-8e54-a5173c1e4d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:43.197806 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-02dfa278-e57b-4bdf-8196-4e48a3e06457 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '097a49d6e8ae4ebaba127e9fc585df2f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:43.207529 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-02dfa278-e57b-4bdf-8196-4e48a3e06457 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '097a49d6e8ae4ebaba127e9fc585df2f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:43.225683 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-02dfa278-e57b-4bdf-8196-4e48a3e06457 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '097a49d6e8ae4ebaba127e9fc585df2f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:43.227443 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-02dfa278-e57b-4bdf-8196-4e48a3e06457 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '097a49d6e8ae4ebaba127e9fc585df2f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:43.235114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 825/1696] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:38:42 2022] DELETE /identity/v3/users/097a49d6e8ae4ebaba127e9fc585df2f => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:38:43.236262 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dfa278-e57b-4bdf-8196-4e48a3e06457 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:43.238749 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-02dfa278-e57b-4bdf-8196-4e48a3e06457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:43.333376 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 872/1697] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:43.337044 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768c4ea-42b7-4172-bc25-5a5b4622f31a None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:43.337606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768c4ea-42b7-4172-bc25-5a5b4622f31a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:43.338096 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768c4ea-42b7-4172-bc25-5a5b4622f31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:43.349540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:43.360154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dfa278-e57b-4bdf-8196-4e48a3e06457 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:43.370427 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-1768c4ea-42b7-4172-bc25-5a5b4622f31a None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:43.370670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96728206-ed49-458c-96b3-0679e2e6abe1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:43.372071 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1768c4ea-42b7-4172-bc25-5a5b4622f31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:43.372188 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96728206-ed49-458c-96b3-0679e2e6abe1 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:43.372354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96728206-ed49-458c-96b3-0679e2e6abe1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:43.372638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96728206-ed49-458c-96b3-0679e2e6abe1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:43.380761 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-96728206-ed49-458c-96b3-0679e2e6abe1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:43.393085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:43.474968 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 873/1698] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:43.476401 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b70b9a-913a-434e-995e-f96a9e5b49ba None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:43.476701 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b70b9a-913a-434e-995e-f96a9e5b49ba None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:43.477519 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b70b9a-913a-434e-995e-f96a9e5b49ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:43.498760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-40b70b9a-913a-434e-995e-f96a9e5b49ba None None] MFA Rules not processed for user `91255c0b2d914ef9be10eb69d7d8ed4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:43.500514 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-40b70b9a-913a-434e-995e-f96a9e5b49ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:43.524575 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96728206-ed49-458c-96b3-0679e2e6abe1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:43.526118 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96728206-ed49-458c-96b3-0679e2e6abe1 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:43.550057 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 826/1699] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:43 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:43.553341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96728206-ed49-458c-96b3-0679e2e6abe1 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:43.558584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-96728206-ed49-458c-96b3-0679e2e6abe1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:43.576685 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 874/1700] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:43.592231 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b70b9a-913a-434e-995e-f96a9e5b49ba None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:43.595298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-40b70b9a-913a-434e-995e-f96a9e5b49ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:43.676588 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96728206-ed49-458c-96b3-0679e2e6abe1 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:43.690800 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a89bed-e574-4891-be8a-b83557c0eaeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:43.691783 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a89bed-e574-4891-be8a-b83557c0eaeb service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:43.692052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a89bed-e574-4891-be8a-b83557c0eaeb service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:43.692335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a89bed-e574-4891-be8a-b83557c0eaeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:43.712565 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a89bed-e574-4891-be8a-b83557c0eaeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:43.714014 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a89bed-e574-4891-be8a-b83557c0eaeb service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:43.735576 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 827/1701] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:43.776519 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b70b9a-913a-434e-995e-f96a9e5b49ba None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:43.793645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1cd38-46c4-48e4-a569-87636dca3661 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:43.794551 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1cd38-46c4-48e4-a569-87636dca3661 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:43.794870 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1cd38-46c4-48e4-a569-87636dca3661 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:43.795337 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1cd38-46c4-48e4-a569-87636dca3661 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:43.819335 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1cd38-46c4-48e4-a569-87636dca3661 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:43.823785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1cd38-46c4-48e4-a569-87636dca3661 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:43.847326 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 875/1702] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:44.255383 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a89bed-e574-4891-be8a-b83557c0eaeb service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:44.257124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b3a89bed-e574-4891-be8a-b83557c0eaeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:44.282312 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a89bed-e574-4891-be8a-b83557c0eaeb service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:44.298878 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205658e7-1f0a-45c9-85ea-1168f5d0134e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:44.299838 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205658e7-1f0a-45c9-85ea-1168f5d0134e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:44.300264 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205658e7-1f0a-45c9-85ea-1168f5d0134e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:44.300549 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205658e7-1f0a-45c9-85ea-1168f5d0134e admin admin] PATH_INFO: `/v3/projects/3263e2a380ae428bb802bad8856e2ba9` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:44.301937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205658e7-1f0a-45c9-85ea-1168f5d0134e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3263e2a380ae428bb802bad8856e2ba9)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:44.311964 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205658e7-1f0a-45c9-85ea-1168f5d0134e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:44.334658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1cd38-46c4-48e4-a569-87636dca3661 service nova] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:44.336349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-16e1cd38-46c4-48e4-a569-87636dca3661 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:44.359819 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1cd38-46c4-48e4-a569-87636dca3661 service nova] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:44.377664 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a20ed-3047-491e-b10e-e0c918a03111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:44.378672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a20ed-3047-491e-b10e-e0c918a03111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:44.378960 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a20ed-3047-491e-b10e-e0c918a03111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:44.379787 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a20ed-3047-491e-b10e-e0c918a03111 admin admin] PATH_INFO: `/v3/users/6052d52bd3cd4b009b51987e12229247` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:44.381080 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a20ed-3047-491e-b10e-e0c918a03111 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6052d52bd3cd4b009b51987e12229247)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:44.384187 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a20ed-3047-491e-b10e-e0c918a03111 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:44.417186 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-205658e7-1f0a-45c9-85ea-1168f5d0134e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3263e2a380ae428bb802bad8856e2ba9', 'request_id': 'req-205658e7-1f0a-45c9-85ea-1168f5d0134e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:44.424218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-205658e7-1f0a-45c9-85ea-1168f5d0134e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3263e2a380ae428bb802bad8856e2ba9', 'request_id': 'req-205658e7-1f0a-45c9-85ea-1168f5d0134e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:44.432347 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-205658e7-1f0a-45c9-85ea-1168f5d0134e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3263e2a380ae428bb802bad8856e2ba9', 'request_id': 'req-205658e7-1f0a-45c9-85ea-1168f5d0134e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:44.453581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-205658e7-1f0a-45c9-85ea-1168f5d0134e admin admin] The token cache is being invalidate because project 3263e2a380ae428bb802bad8856e2ba9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:38:44.453963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-205658e7-1f0a-45c9-85ea-1168f5d0134e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:44.454667 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-205658e7-1f0a-45c9-85ea-1168f5d0134e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:44.456869 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 828/1703] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:38:44 2022] DELETE /identity/v3/projects/3263e2a380ae428bb802bad8856e2ba9 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:44.460992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205658e7-1f0a-45c9-85ea-1168f5d0134e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:44.463614 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-205658e7-1f0a-45c9-85ea-1168f5d0134e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:44.467656 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:44.474316 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-119a20ed-3047-491e-b10e-e0c918a03111 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6052d52bd3cd4b009b51987e12229247'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:44.491505 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-119a20ed-3047-491e-b10e-e0c918a03111 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6052d52bd3cd4b009b51987e12229247'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:44.492655 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-119a20ed-3047-491e-b10e-e0c918a03111 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6052d52bd3cd4b009b51987e12229247'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:44.508093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:44.517763 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-119a20ed-3047-491e-b10e-e0c918a03111 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6052d52bd3cd4b009b51987e12229247'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:44.518517 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-119a20ed-3047-491e-b10e-e0c918a03111 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6052d52bd3cd4b009b51987e12229247'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:44.551915 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 876/1704] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:38:44 2022] DELETE /identity/v3/users/6052d52bd3cd4b009b51987e12229247 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:44.553776 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04819127-47e1-483e-8c69-ec5209a46925 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:44.554486 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04819127-47e1-483e-8c69-ec5209a46925 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:44.556037 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04819127-47e1-483e-8c69-ec5209a46925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:44.579721 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-04819127-47e1-483e-8c69-ec5209a46925 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:44.581115 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-04819127-47e1-483e-8c69-ec5209a46925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:44.593751 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205658e7-1f0a-45c9-85ea-1168f5d0134e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:44.694692 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 877/1705] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:44.696065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8e5be-1dee-49d4-8da5-56d12a44ea88 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:44.696355 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8e5be-1dee-49d4-8da5-56d12a44ea88 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:44.696631 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8e5be-1dee-49d4-8da5-56d12a44ea88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:44.783207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3a603-43f1-42de-95d3-22482bb804b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:44.783407 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-c5d8e5be-1dee-49d4-8da5-56d12a44ea88 None None] MFA Rules not processed for user `a36fb9afb5d54d3493e8752bae88dd13`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:44.784650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3a603-43f1-42de-95d3-22482bb804b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:44.784769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c5d8e5be-1dee-49d4-8da5-56d12a44ea88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:44.785244 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3a603-43f1-42de-95d3-22482bb804b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:44.785719 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3a603-43f1-42de-95d3-22482bb804b6 admin admin] PATH_INFO: `/v3/users/508c3f83bddf4c38b3c09ffbe6f41214` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:44.787335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e3a603-43f1-42de-95d3-22482bb804b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=508c3f83bddf4c38b3c09ffbe6f41214)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:44.790408 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e3a603-43f1-42de-95d3-22482bb804b6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:44.866924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54e3a603-43f1-42de-95d3-22482bb804b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '508c3f83bddf4c38b3c09ffbe6f41214'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:44.878193 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54e3a603-43f1-42de-95d3-22482bb804b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '508c3f83bddf4c38b3c09ffbe6f41214'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:44.892371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54e3a603-43f1-42de-95d3-22482bb804b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '508c3f83bddf4c38b3c09ffbe6f41214'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:44.900513 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 878/1706] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:44.901658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8e5be-1dee-49d4-8da5-56d12a44ea88 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:44.903627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c5d8e5be-1dee-49d4-8da5-56d12a44ea88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:44.905527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54e3a603-43f1-42de-95d3-22482bb804b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '508c3f83bddf4c38b3c09ffbe6f41214'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:44.906787 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54e3a603-43f1-42de-95d3-22482bb804b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '508c3f83bddf4c38b3c09ffbe6f41214'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:44.921165 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 829/1707] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:38:44 2022] DELETE /identity/v3/users/508c3f83bddf4c38b3c09ffbe6f41214 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:38:44.922677 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712651d7-0612-4922-9b32-7f358a6801e2 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:44.922954 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712651d7-0612-4922-9b32-7f358a6801e2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:44.923261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712651d7-0612-4922-9b32-7f358a6801e2 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:44.936930 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 830/1708] 10.222.0.67 () {54 vars in 912 bytes} [Tue May 31 14:38:44 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:38:44.937887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712651d7-0612-4922-9b32-7f358a6801e2 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:44.944426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-712651d7-0612-4922-9b32-7f358a6801e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:45.011650 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:45.019836 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8e5be-1dee-49d4-8da5-56d12a44ea88 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:45.022775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:45.076916 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712651d7-0612-4922-9b32-7f358a6801e2 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:45.091222 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2549e-bbe2-4fa5-b706-2d94bc3d114e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:45.094102 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2549e-bbe2-4fa5-b706-2d94bc3d114e service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:45.095712 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2549e-bbe2-4fa5-b706-2d94bc3d114e service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:45.096057 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2549e-bbe2-4fa5-b706-2d94bc3d114e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:45.100116 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8ee2549e-bbe2-4fa5-b706-2d94bc3d114e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:45.134394 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224999f2-dfd7-4b28-af24-9e925df1e116 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:45.136174 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224999f2-dfd7-4b28-af24-9e925df1e116 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:45.136597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224999f2-dfd7-4b28-af24-9e925df1e116 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:45.137062 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224999f2-dfd7-4b28-af24-9e925df1e116 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:45.147236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-224999f2-dfd7-4b28-af24-9e925df1e116 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:45.274081 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2549e-bbe2-4fa5-b706-2d94bc3d114e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:45.275924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2549e-bbe2-4fa5-b706-2d94bc3d114e service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:45.291582 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224999f2-dfd7-4b28-af24-9e925df1e116 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:45.296349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224999f2-dfd7-4b28-af24-9e925df1e116 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:45.300901 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 831/1709] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:45.304196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca2fe4-1247-41be-acf6-9952b6ea56fb None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:45.308634 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca2fe4-1247-41be-acf6-9952b6ea56fb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:45.309204 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca2fe4-1247-41be-acf6-9952b6ea56fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:45.342416 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 879/1710] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:45.368481 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7dca2fe4-1247-41be-acf6-9952b6ea56fb None None] MFA Rules not processed for user `8966bc2ea7ff497c84214c9c9e6d53a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:45.372136 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7dca2fe4-1247-41be-acf6-9952b6ea56fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:45.399174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 832/1711] 10.222.0.67 () {58 vars in 1009 bytes} [Tue May 31 14:38:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:45.417499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224999f2-dfd7-4b28-af24-9e925df1e116 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:45.422893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-224999f2-dfd7-4b28-af24-9e925df1e116 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:45.442388 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224999f2-dfd7-4b28-af24-9e925df1e116 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:45.454356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff0356-1e9a-4aca-99b9-13298e26d0cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:45.477221 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff0356-1e9a-4aca-99b9-13298e26d0cf service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:45.477511 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff0356-1e9a-4aca-99b9-13298e26d0cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:45.477789 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff0356-1e9a-4aca-99b9-13298e26d0cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:45.496603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff0356-1e9a-4aca-99b9-13298e26d0cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:45.499405 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff0356-1e9a-4aca-99b9-13298e26d0cf service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:45.530298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 880/1712] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:46.090464 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca2fe4-1247-41be-acf6-9952b6ea56fb None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:46.094236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7dca2fe4-1247-41be-acf6-9952b6ea56fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:46.109189 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca2fe4-1247-41be-acf6-9952b6ea56fb None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:46.125968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd7240-eeb2-4d32-89af-ff8cc16f03b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:46.127143 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd7240-eeb2-4d32-89af-ff8cc16f03b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:46.127541 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd7240-eeb2-4d32-89af-ff8cc16f03b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:46.127941 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd7240-eeb2-4d32-89af-ff8cc16f03b7 admin admin] PATH_INFO: `/v3/projects/95da8209d46f4d8683befcd8b59305d2` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:46.130478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd7240-eeb2-4d32-89af-ff8cc16f03b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95da8209d46f4d8683befcd8b59305d2)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:46.135029 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd7240-eeb2-4d32-89af-ff8cc16f03b7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:46.173664 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff0356-1e9a-4aca-99b9-13298e26d0cf service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:46.175139 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d0ff0356-1e9a-4aca-99b9-13298e26d0cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:46.214851 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff0356-1e9a-4aca-99b9-13298e26d0cf service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:46.231473 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d957b1-fef0-4f03-baae-8e1d7f55ffb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:46.232468 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d957b1-fef0-4f03-baae-8e1d7f55ffb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:46.232753 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d957b1-fef0-4f03-baae-8e1d7f55ffb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:46.233016 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d957b1-fef0-4f03-baae-8e1d7f55ffb3 admin admin] PATH_INFO: `/v3/projects/b34f1f2dc22a4dea9dd1129c234ba55c` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:46.234600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d957b1-fef0-4f03-baae-8e1d7f55ffb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b34f1f2dc22a4dea9dd1129c234ba55c)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:46.236676 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d957b1-fef0-4f03-baae-8e1d7f55ffb3 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:46.276478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-33fd7240-eeb2-4d32-89af-ff8cc16f03b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95da8209d46f4d8683befcd8b59305d2', 'request_id': 'req-33fd7240-eeb2-4d32-89af-ff8cc16f03b7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:46.294343 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-33fd7240-eeb2-4d32-89af-ff8cc16f03b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95da8209d46f4d8683befcd8b59305d2', 'request_id': 'req-33fd7240-eeb2-4d32-89af-ff8cc16f03b7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:46.311276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-33fd7240-eeb2-4d32-89af-ff8cc16f03b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95da8209d46f4d8683befcd8b59305d2', 'request_id': 'req-33fd7240-eeb2-4d32-89af-ff8cc16f03b7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:46.323042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b7d957b1-fef0-4f03-baae-8e1d7f55ffb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b34f1f2dc22a4dea9dd1129c234ba55c', 'request_id': 'req-b7d957b1-fef0-4f03-baae-8e1d7f55ffb3'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:46.334343 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b7d957b1-fef0-4f03-baae-8e1d7f55ffb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b34f1f2dc22a4dea9dd1129c234ba55c', 'request_id': 'req-b7d957b1-fef0-4f03-baae-8e1d7f55ffb3'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:46.345610 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-33fd7240-eeb2-4d32-89af-ff8cc16f03b7 admin admin] The token cache is being invalidate because project 95da8209d46f4d8683befcd8b59305d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:38:46.346003 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-33fd7240-eeb2-4d32-89af-ff8cc16f03b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:46.347106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-33fd7240-eeb2-4d32-89af-ff8cc16f03b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:46.350223 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 833/1713] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:38:46 2022] DELETE /identity/v3/projects/95da8209d46f4d8683befcd8b59305d2 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:38:46.355663 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd7240-eeb2-4d32-89af-ff8cc16f03b7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:46.359285 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-33fd7240-eeb2-4d32-89af-ff8cc16f03b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:46.362482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b7d957b1-fef0-4f03-baae-8e1d7f55ffb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b34f1f2dc22a4dea9dd1129c234ba55c', 'request_id': 'req-b7d957b1-fef0-4f03-baae-8e1d7f55ffb3'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:46.395833 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b7d957b1-fef0-4f03-baae-8e1d7f55ffb3 admin admin] The token cache is being invalidate because project b34f1f2dc22a4dea9dd1129c234ba55c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:38:46.396486 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b7d957b1-fef0-4f03-baae-8e1d7f55ffb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:46.397377 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b7d957b1-fef0-4f03-baae-8e1d7f55ffb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:46.399909 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 881/1714] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:38:46 2022] DELETE /identity/v3/projects/b34f1f2dc22a4dea9dd1129c234ba55c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:46.406036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6dcc8-a39d-4a07-a476-e6da39da33b6 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:46.415294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6dcc8-a39d-4a07-a476-e6da39da33b6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:46.415665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6dcc8-a39d-4a07-a476-e6da39da33b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:46.440772 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-59f6dcc8-a39d-4a07-a476-e6da39da33b6 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:46.443732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-59f6dcc8-a39d-4a07-a476-e6da39da33b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:46.502445 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:46.519512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd7240-eeb2-4d32-89af-ff8cc16f03b7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:46.526731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:46.653046 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 882/1715] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:38:46.661066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de40b1-6dc5-4efb-8966-cc87ee20bcb6 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:46.675474 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de40b1-6dc5-4efb-8966-cc87ee20bcb6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:46.676007 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de40b1-6dc5-4efb-8966-cc87ee20bcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:46.682141 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc236b62-b0bf-45e5-9a4d-6beeb8b80665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:46.683404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc236b62-b0bf-45e5-9a4d-6beeb8b80665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:46.683723 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc236b62-b0bf-45e5-9a4d-6beeb8b80665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:46.684002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc236b62-b0bf-45e5-9a4d-6beeb8b80665 admin admin] PATH_INFO: `/v3/users/35cbfe2de26143e18e6bc10ffdc6cc55` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:46.689377 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc236b62-b0bf-45e5-9a4d-6beeb8b80665 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35cbfe2de26143e18e6bc10ffdc6cc55)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:46.692281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc236b62-b0bf-45e5-9a4d-6beeb8b80665 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:46.764895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-24de40b1-6dc5-4efb-8966-cc87ee20bcb6 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:46.766505 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-24de40b1-6dc5-4efb-8966-cc87ee20bcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:46.783371 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 883/1716] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:46.787528 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de40b1-6dc5-4efb-8966-cc87ee20bcb6 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:46.829321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de40b1-6dc5-4efb-8966-cc87ee20bcb6 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:46.852820 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55a63f-9d29-4ea8-aae0-dd896bd7e2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:46.853565 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55a63f-9d29-4ea8-aae0-dd896bd7e2fa admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:46.859204 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55a63f-9d29-4ea8-aae0-dd896bd7e2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:46.859496 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55a63f-9d29-4ea8-aae0-dd896bd7e2fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:46.865468 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55a63f-9d29-4ea8-aae0-dd896bd7e2fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:38:46.865785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55a63f-9d29-4ea8-aae0-dd896bd7e2fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:46.869416 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55a63f-9d29-4ea8-aae0-dd896bd7e2fa admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:46.884792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cc236b62-b0bf-45e5-9a4d-6beeb8b80665 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35cbfe2de26143e18e6bc10ffdc6cc55'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:46.899041 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cc236b62-b0bf-45e5-9a4d-6beeb8b80665 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35cbfe2de26143e18e6bc10ffdc6cc55'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:46.905876 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 884/1717] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:38:46.912041 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cf239-7dfb-4d7a-821a-10b7774c2ee3 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:46.912366 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cf239-7dfb-4d7a-821a-10b7774c2ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:46.912652 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cf239-7dfb-4d7a-821a-10b7774c2ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:46.917549 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cc236b62-b0bf-45e5-9a4d-6beeb8b80665 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35cbfe2de26143e18e6bc10ffdc6cc55'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:46.950013 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cc236b62-b0bf-45e5-9a4d-6beeb8b80665 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35cbfe2de26143e18e6bc10ffdc6cc55'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:46.951038 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cc236b62-b0bf-45e5-9a4d-6beeb8b80665 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35cbfe2de26143e18e6bc10ffdc6cc55'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:46.965546 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-fa8cf239-7dfb-4d7a-821a-10b7774c2ee3 None None] MFA Rules not processed for user `91255c0b2d914ef9be10eb69d7d8ed4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:46.966859 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-fa8cf239-7dfb-4d7a-821a-10b7774c2ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:46.970612 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 834/1718] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:38:46 2022] DELETE /identity/v3/users/35cbfe2de26143e18e6bc10ffdc6cc55 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:46.972772 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc236b62-b0bf-45e5-9a4d-6beeb8b80665 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:46.976043 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cc236b62-b0bf-45e5-9a4d-6beeb8b80665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:47.148838 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc236b62-b0bf-45e5-9a4d-6beeb8b80665 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:47.152980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 885/1719] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:47.193505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ab0de-fd4b-4eb0-8c4a-c60d8eb38813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:47.196065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ab0de-fd4b-4eb0-8c4a-c60d8eb38813 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:47.196357 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ab0de-fd4b-4eb0-8c4a-c60d8eb38813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:47.197388 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ab0de-fd4b-4eb0-8c4a-c60d8eb38813 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:47.211512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ab0de-fd4b-4eb0-8c4a-c60d8eb38813 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:47.215817 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ab0de-fd4b-4eb0-8c4a-c60d8eb38813 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:47.341318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 835/1720] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:38:46 2022] POST /identity/v3/projects => generated 389 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:47.345850 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8cf239-7dfb-4d7a-821a-10b7774c2ee3 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:47.366302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8cf239-7dfb-4d7a-821a-10b7774c2ee3 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:47.505131 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a63bb9-8448-4c7a-bbbd-9a026dc56bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:47.506258 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a63bb9-8448-4c7a-bbbd-9a026dc56bf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:47.506258 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a63bb9-8448-4c7a-bbbd-9a026dc56bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:47.506530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a63bb9-8448-4c7a-bbbd-9a026dc56bf1 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:47.507237 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a63bb9-8448-4c7a-bbbd-9a026dc56bf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:47.509777 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a63bb9-8448-4c7a-bbbd-9a026dc56bf1 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:47.551693 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 886/1721] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:38:47 2022] POST /identity/v3/users => generated 295 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:47.556814 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ab0de-fd4b-4eb0-8c4a-c60d8eb38813 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:47.589249 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ab0de-fd4b-4eb0-8c4a-c60d8eb38813 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:47.597562 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de8009-78ff-42ee-b0e3-ba2444594374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:47.598616 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de8009-78ff-42ee-b0e3-ba2444594374 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:47.598931 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de8009-78ff-42ee-b0e3-ba2444594374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:47.602676 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de8009-78ff-42ee-b0e3-ba2444594374 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:47.602676 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de8009-78ff-42ee-b0e3-ba2444594374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:47.602676 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de8009-78ff-42ee-b0e3-ba2444594374 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:47.615338 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a63bb9-8448-4c7a-bbbd-9a026dc56bf1 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:47.619849 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f3a63bb9-8448-4c7a-bbbd-9a026dc56bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:47.621634 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 836/1722] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:47.625719 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de8009-78ff-42ee-b0e3-ba2444594374 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:47.648275 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:47.663124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de8009-78ff-42ee-b0e3-ba2444594374 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:47.667800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a63bb9-8448-4c7a-bbbd-9a026dc56bf1 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:47.682410 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d3ca0-4d4a-4825-ab1f-5df121aa23ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:47.687029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d3ca0-4d4a-4825-ab1f-5df121aa23ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:47.687387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d3ca0-4d4a-4825-ab1f-5df121aa23ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:47.687881 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d3ca0-4d4a-4825-ab1f-5df121aa23ea admin admin] PATH_INFO: `/v3/projects/656565f2f9c64cbf9c3c33388ffb779e` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:47.691222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d3ca0-4d4a-4825-ab1f-5df121aa23ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=656565f2f9c64cbf9c3c33388ffb779e)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:47.692329 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4637e-c420-409f-b746-c84a383729e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:47.696258 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4637e-c420-409f-b746-c84a383729e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:47.698531 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4637e-c420-409f-b746-c84a383729e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:47.698971 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4637e-c420-409f-b746-c84a383729e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:47.703062 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4637e-c420-409f-b746-c84a383729e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:47.705132 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d3ca0-4d4a-4825-ab1f-5df121aa23ea admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:47.709020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4637e-c420-409f-b746-c84a383729e0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:47.745826 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 837/1723] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:47.752541 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4637e-c420-409f-b746-c84a383729e0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:47.804307 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4637e-c420-409f-b746-c84a383729e0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:47.821724 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788d38a-a2de-4f3f-9bd9-1b15598772d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:47.822671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788d38a-a2de-4f3f-9bd9-1b15598772d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:47.822974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788d38a-a2de-4f3f-9bd9-1b15598772d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:47.823255 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788d38a-a2de-4f3f-9bd9-1b15598772d6 admin admin] PATH_INFO: `/v3/projects/97bb9f0a317d46dbbbca44298eda83e4/users/b8774b57af854d47912a4468aaf94677/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:47.862880 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:47.900168 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788d38a-a2de-4f3f-9bd9-1b15598772d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97bb9f0a317d46dbbbca44298eda83e4, user_id=b8774b57af854d47912a4468aaf94677, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:47.908496 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788d38a-a2de-4f3f-9bd9-1b15598772d6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:47.911241 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-718d3ca0-4d4a-4825-ab1f-5df121aa23ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '656565f2f9c64cbf9c3c33388ffb779e', 'request_id': 'req-718d3ca0-4d4a-4825-ab1f-5df121aa23ea'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:47.928505 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-718d3ca0-4d4a-4825-ab1f-5df121aa23ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '656565f2f9c64cbf9c3c33388ffb779e', 'request_id': 'req-718d3ca0-4d4a-4825-ab1f-5df121aa23ea'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:47.944031 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 838/1724] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:47 2022] PUT /identity/v3/projects/97bb9f0a317d46dbbbca44298eda83e4/users/b8774b57af854d47912a4468aaf94677/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:47.945009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-718d3ca0-4d4a-4825-ab1f-5df121aa23ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '656565f2f9c64cbf9c3c33388ffb779e', 'request_id': 'req-718d3ca0-4d4a-4825-ab1f-5df121aa23ea'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:47.954173 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8d059-df8d-4677-8093-42ae78c5c232 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:47.954704 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8d059-df8d-4677-8093-42ae78c5c232 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:47.954807 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8d059-df8d-4677-8093-42ae78c5c232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:47.980331 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-718d3ca0-4d4a-4825-ab1f-5df121aa23ea admin admin] The token cache is being invalidate because project 656565f2f9c64cbf9c3c33388ffb779e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:38:47.981897 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-718d3ca0-4d4a-4825-ab1f-5df121aa23ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:47.983667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-718d3ca0-4d4a-4825-ab1f-5df121aa23ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:48.005472 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 887/1725] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:38:47 2022] DELETE /identity/v3/projects/656565f2f9c64cbf9c3c33388ffb779e => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:48.007867 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4db8d059-df8d-4677-8093-42ae78c5c232 None None] MFA Rules not processed for user `b8774b57af854d47912a4468aaf94677`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:48.010085 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4db8d059-df8d-4677-8093-42ae78c5c232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:48.010711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1896e4-a161-4aa7-b5a3-6997f34fe18b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:48.011140 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1896e4-a161-4aa7-b5a3-6997f34fe18b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:48.011468 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1896e4-a161-4aa7-b5a3-6997f34fe18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:48.064547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-de1896e4-a161-4aa7-b5a3-6997f34fe18b None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:48.069783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-de1896e4-a161-4aa7-b5a3-6997f34fe18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:48.204001 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 839/1726] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:38:48.205267 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8d059-df8d-4677-8093-42ae78c5c232 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:48.211619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 888/1727] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:48.213677 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4db8d059-df8d-4677-8093-42ae78c5c232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:48.219822 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea1207-bc0f-4f1a-beaa-a35c97824bc2 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:48.222337 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea1207-bc0f-4f1a-beaa-a35c97824bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:48.222976 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea1207-bc0f-4f1a-beaa-a35c97824bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:48.260651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-c8ea1207-bc0f-4f1a-beaa-a35c97824bc2 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:48.262697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c8ea1207-bc0f-4f1a-beaa-a35c97824bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:48.268189 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8d059-df8d-4677-8093-42ae78c5c232 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:48.279011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023831e-2614-4602-9c71-18afb2dd7142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:48.279700 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023831e-2614-4602-9c71-18afb2dd7142 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:48.279983 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023831e-2614-4602-9c71-18afb2dd7142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:48.280261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023831e-2614-4602-9c71-18afb2dd7142 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:48.280971 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8023831e-2614-4602-9c71-18afb2dd7142 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:48.285300 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8023831e-2614-4602-9c71-18afb2dd7142 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:48.285462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 889/1728] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:48.289674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea1207-bc0f-4f1a-beaa-a35c97824bc2 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:48.304174 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea1207-bc0f-4f1a-beaa-a35c97824bc2 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:48.313602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7fce6-2927-4e94-80a2-db8b2c6c846b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:48.314413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7fce6-2927-4e94-80a2-db8b2c6c846b admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:48.314917 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7fce6-2927-4e94-80a2-db8b2c6c846b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:48.315054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7fce6-2927-4e94-80a2-db8b2c6c846b admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:48.321302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7fce6-2927-4e94-80a2-db8b2c6c846b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:38:48.321302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7fce6-2927-4e94-80a2-db8b2c6c846b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:48.321302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7fce6-2927-4e94-80a2-db8b2c6c846b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:48.326259 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 840/1729] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:38:48 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:48.327200 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023831e-2614-4602-9c71-18afb2dd7142 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:48.354232 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 890/1730] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:38:48.358209 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7fce6-2927-4e94-80a2-db8b2c6c846b admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:48.378096 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7fce6-2927-4e94-80a2-db8b2c6c846b admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:48.426523 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023831e-2614-4602-9c71-18afb2dd7142 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:48.433627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db750ae-e213-4bc4-abd0-b403e7d45116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:48.434563 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db750ae-e213-4bc4-abd0-b403e7d45116 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:48.434989 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db750ae-e213-4bc4-abd0-b403e7d45116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:48.435411 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db750ae-e213-4bc4-abd0-b403e7d45116 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:48.436221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db750ae-e213-4bc4-abd0-b403e7d45116 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:48.439533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db750ae-e213-4bc4-abd0-b403e7d45116 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:48.469271 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c14aab-5859-48e0-9fed-b80dd2b8a624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:48.469967 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c14aab-5859-48e0-9fed-b80dd2b8a624 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:48.470255 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c14aab-5859-48e0-9fed-b80dd2b8a624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:48.470536 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c14aab-5859-48e0-9fed-b80dd2b8a624 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:48.471326 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c14aab-5859-48e0-9fed-b80dd2b8a624 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:48.473812 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c14aab-5859-48e0-9fed-b80dd2b8a624 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:48.480702 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 841/1731] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:38:48 2022] POST /identity/v3/users => generated 300 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:48.481996 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db750ae-e213-4bc4-abd0-b403e7d45116 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:48.494451 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db750ae-e213-4bc4-abd0-b403e7d45116 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:48.504370 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e873df-87a6-4b31-a765-c4a3361eb0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:48.505320 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e873df-87a6-4b31-a765-c4a3361eb0e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:48.505750 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e873df-87a6-4b31-a765-c4a3361eb0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:48.506171 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e873df-87a6-4b31-a765-c4a3361eb0e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:48.507237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e873df-87a6-4b31-a765-c4a3361eb0e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:48.509861 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e873df-87a6-4b31-a765-c4a3361eb0e2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:48.513706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 891/1732] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:38:48 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:48.517246 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c14aab-5859-48e0-9fed-b80dd2b8a624 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:48.528687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 842/1733] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:48.533055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e873df-87a6-4b31-a765-c4a3361eb0e2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:48.554034 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e873df-87a6-4b31-a765-c4a3361eb0e2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:48.554334 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:48.581410 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:48.629066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c14aab-5859-48e0-9fed-b80dd2b8a624 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:48.637506 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19214e3-a108-4f93-af4b-ee9693ec2e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:48.638202 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19214e3-a108-4f93-af4b-ee9693ec2e93 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:48.638802 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19214e3-a108-4f93-af4b-ee9693ec2e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:48.639358 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19214e3-a108-4f93-af4b-ee9693ec2e93 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:48.640077 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19214e3-a108-4f93-af4b-ee9693ec2e93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:48.644027 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19214e3-a108-4f93-af4b-ee9693ec2e93 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:48.646850 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee87c75-e7ef-4def-a219-c89b4f6e08d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:48.647863 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee87c75-e7ef-4def-a219-c89b4f6e08d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:48.648297 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee87c75-e7ef-4def-a219-c89b4f6e08d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:48.648714 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee87c75-e7ef-4def-a219-c89b4f6e08d3 admin admin] PATH_INFO: `/v3/projects/51ee47b55853470f859c88fa2fb01ee3/users/0c843397485f4facbf3fbd98134f3eb9/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:48.669070 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee87c75-e7ef-4def-a219-c89b4f6e08d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51ee47b55853470f859c88fa2fb01ee3, user_id=0c843397485f4facbf3fbd98134f3eb9, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:48.672448 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee87c75-e7ef-4def-a219-c89b4f6e08d3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:48.683304 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 892/1734] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:38:48 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:38:48.684959 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19214e3-a108-4f93-af4b-ee9693ec2e93 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:48.695775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 843/1735] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:48 2022] PUT /identity/v3/projects/51ee47b55853470f859c88fa2fb01ee3/users/0c843397485f4facbf3fbd98134f3eb9/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:38:48.697095 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee87c75-e7ef-4def-a219-c89b4f6e08d3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:48.700677 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19214e3-a108-4f93-af4b-ee9693ec2e93 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:48.774251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b0f05-5cb5-4e87-91f4-64136f5b2315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:48.775085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b0f05-5cb5-4e87-91f4-64136f5b2315 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:48.775748 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b0f05-5cb5-4e87-91f4-64136f5b2315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:48.775748 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b0f05-5cb5-4e87-91f4-64136f5b2315 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:48.776569 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b0f05-5cb5-4e87-91f4-64136f5b2315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:48.778506 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b0f05-5cb5-4e87-91f4-64136f5b2315 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:48.788589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee87c75-e7ef-4def-a219-c89b4f6e08d3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:48.795453 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fcf53b-317f-46ad-8ad9-c47b9a797794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:48.796278 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcf53b-317f-46ad-8ad9-c47b9a797794 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:48.796630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcf53b-317f-46ad-8ad9-c47b9a797794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:48.797104 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcf53b-317f-46ad-8ad9-c47b9a797794 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:48.798051 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fcf53b-317f-46ad-8ad9-c47b9a797794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:48.799902 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fcf53b-317f-46ad-8ad9-c47b9a797794 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:48.800079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 893/1736] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:48.805850 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b0f05-5cb5-4e87-91f4-64136f5b2315 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:48.819103 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 844/1737] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:48.828724 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b0f05-5cb5-4e87-91f4-64136f5b2315 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:48.834394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fcf53b-317f-46ad-8ad9-c47b9a797794 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:48.839387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a60f0-f077-4e58-a8e1-f8131fb274ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:48.840228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a60f0-f077-4e58-a8e1-f8131fb274ee admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:48.840617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a60f0-f077-4e58-a8e1-f8131fb274ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:48.840992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a60f0-f077-4e58-a8e1-f8131fb274ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:48.841829 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a60f0-f077-4e58-a8e1-f8131fb274ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:48.844313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a60f0-f077-4e58-a8e1-f8131fb274ee admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:48.850421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fcf53b-317f-46ad-8ad9-c47b9a797794 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:48.865039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 894/1738] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:48.866029 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e1079-ba0a-463c-bd47-6896925d856f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:48.866999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e1079-ba0a-463c-bd47-6896925d856f admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:48.867277 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e1079-ba0a-463c-bd47-6896925d856f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:48.867596 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e1079-ba0a-463c-bd47-6896925d856f admin admin] PATH_INFO: `/v3/projects/51ee47b55853470f859c88fa2fb01ee3/users/0c843397485f4facbf3fbd98134f3eb9/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:48.870206 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a60f0-f077-4e58-a8e1-f8131fb274ee admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:48.878564 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e1079-ba0a-463c-bd47-6896925d856f admin admin] RBAC: Authorizing `identity:create_grant(project_id=51ee47b55853470f859c88fa2fb01ee3, user_id=0c843397485f4facbf3fbd98134f3eb9, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:48.882189 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e1079-ba0a-463c-bd47-6896925d856f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:48.890078 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a60f0-f077-4e58-a8e1-f8131fb274ee admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:48.895875 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 845/1739] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:48 2022] PUT /identity/v3/projects/51ee47b55853470f859c88fa2fb01ee3/users/0c843397485f4facbf3fbd98134f3eb9/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:48.909814 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8a730-fb55-42a0-a0a5-03a3649283dd None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:48.910241 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8a730-fb55-42a0-a0a5-03a3649283dd None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:48.910705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8a730-fb55-42a0-a0a5-03a3649283dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:49.002423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-65b8a730-fb55-42a0-a0a5-03a3649283dd None None] MFA Rules not processed for user `0c843397485f4facbf3fbd98134f3eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:49.004401 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-65b8a730-fb55-42a0-a0a5-03a3649283dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:49.010751 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062809d5-95e1-4992-a915-96037479d391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:49.011699 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062809d5-95e1-4992-a915-96037479d391 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:49.011987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062809d5-95e1-4992-a915-96037479d391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:49.012266 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062809d5-95e1-4992-a915-96037479d391 admin admin] PATH_INFO: `/v3/projects/6a12403ef5d54834b76c69b1315174a3/users/07826c2e4709491eb92a53473e02de73/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:49.032432 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062809d5-95e1-4992-a915-96037479d391 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a12403ef5d54834b76c69b1315174a3, user_id=07826c2e4709491eb92a53473e02de73, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:49.036703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062809d5-95e1-4992-a915-96037479d391 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:49.065964 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 895/1740] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:48 2022] PUT /identity/v3/projects/6a12403ef5d54834b76c69b1315174a3/users/07826c2e4709491eb92a53473e02de73/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:49.077420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31dd5d-4fa4-4d7b-a48b-273a1fd3cbc7 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:49.078075 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31dd5d-4fa4-4d7b-a48b-273a1fd3cbc7 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:49.078453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31dd5d-4fa4-4d7b-a48b-273a1fd3cbc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:49.101549 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 846/1741] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:49.113365 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8a730-fb55-42a0-a0a5-03a3649283dd None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:49.116245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-65b8a730-fb55-42a0-a0a5-03a3649283dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:49.120991 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-4d31dd5d-4fa4-4d7b-a48b-273a1fd3cbc7 None None] MFA Rules not processed for user `07826c2e4709491eb92a53473e02de73`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:49.127108 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4d31dd5d-4fa4-4d7b-a48b-273a1fd3cbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:49.136029 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:49.147312 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:49.209869 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8a730-fb55-42a0-a0a5-03a3649283dd None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:49.282457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 896/1742] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:49.286832 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d31dd5d-4fa4-4d7b-a48b-273a1fd3cbc7 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:49.300840 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67bf69-0c15-4341-abb3-888a621b7cfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:49.301907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67bf69-0c15-4341-abb3-888a621b7cfe service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:49.302334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67bf69-0c15-4341-abb3-888a621b7cfe service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:49.302753 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67bf69-0c15-4341-abb3-888a621b7cfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:49.407841 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d31dd5d-4fa4-4d7b-a48b-273a1fd3cbc7 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:49.422189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbd5ca-9fd6-4ca2-829c-08f7b2a7c4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:49.422953 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbd5ca-9fd6-4ca2-829c-08f7b2a7c4fe admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:49.423248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbd5ca-9fd6-4ca2-829c-08f7b2a7c4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:49.423530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbd5ca-9fd6-4ca2-829c-08f7b2a7c4fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:49.425618 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acbd5ca-9fd6-4ca2-829c-08f7b2a7c4fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:49.427956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acbd5ca-9fd6-4ca2-829c-08f7b2a7c4fe admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:49.434430 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a67bf69-0c15-4341-abb3-888a621b7cfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:49.438034 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a67bf69-0c15-4341-abb3-888a621b7cfe service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:49.451517 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 847/1743] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:49.477248 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67bf69-0c15-4341-abb3-888a621b7cfe service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:49.491595 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 897/1744] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:38:49 2022] POST /identity/v3/projects => generated 381 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:49.496302 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67bf69-0c15-4341-abb3-888a621b7cfe service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:49.598188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2a7b9-42b8-4782-bb49-64f9c41ddc95 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:49.599676 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2a7b9-42b8-4782-bb49-64f9c41ddc95 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:49.600061 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2a7b9-42b8-4782-bb49-64f9c41ddc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:49.657239 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-85c2a7b9-42b8-4782-bb49-64f9c41ddc95 None None] MFA Rules not processed for user `a36fb9afb5d54d3493e8752bae88dd13`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:49.657239 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-85c2a7b9-42b8-4782-bb49-64f9c41ddc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:49.667397 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb836342-de27-49ee-ac0a-6fa85f9359b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:49.671092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb836342-de27-49ee-ac0a-6fa85f9359b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:49.672986 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb836342-de27-49ee-ac0a-6fa85f9359b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:49.674317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb836342-de27-49ee-ac0a-6fa85f9359b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:49.677198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb836342-de27-49ee-ac0a-6fa85f9359b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:49.682943 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb836342-de27-49ee-ac0a-6fa85f9359b4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:49.747781 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:49.773280 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 848/1745] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:38:49 2022] POST /identity/v3/users => generated 297 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:38:49.774787 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb836342-de27-49ee-ac0a-6fa85f9359b4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:49.801886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb836342-de27-49ee-ac0a-6fa85f9359b4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:49.813883 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa02a5-b44e-462c-b521-8c9ee00a3c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:49.815452 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa02a5-b44e-462c-b521-8c9ee00a3c0e admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:49.817208 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa02a5-b44e-462c-b521-8c9ee00a3c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:49.817580 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa02a5-b44e-462c-b521-8c9ee00a3c0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:49.819906 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa02a5-b44e-462c-b521-8c9ee00a3c0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:49.822600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa02a5-b44e-462c-b521-8c9ee00a3c0e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:49.857744 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 898/1746] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:49.859167 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2a7b9-42b8-4782-bb49-64f9c41ddc95 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:49.859455 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 849/1747] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:49.875225 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa02a5-b44e-462c-b521-8c9ee00a3c0e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:49.881137 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2a7b9-42b8-4782-bb49-64f9c41ddc95 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:49.886922 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15de5c6-a62e-4070-8921-a5d099386ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:49.887933 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15de5c6-a62e-4070-8921-a5d099386ee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:49.888350 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15de5c6-a62e-4070-8921-a5d099386ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:49.888635 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15de5c6-a62e-4070-8921-a5d099386ee3 admin admin] PATH_INFO: `/v3/projects/2c447b607152422aa828c2689eb0b824/users/e8d596b8e5f445ab8dc8bee1ea5fcb00/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:49.916994 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15de5c6-a62e-4070-8921-a5d099386ee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c447b607152422aa828c2689eb0b824, user_id=e8d596b8e5f445ab8dc8bee1ea5fcb00, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:49.924934 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15de5c6-a62e-4070-8921-a5d099386ee3 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:49.932598 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa02a5-b44e-462c-b521-8c9ee00a3c0e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:49.943267 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1527ddf4-5897-4bf4-8fa6-791913eac2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:49.944712 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527ddf4-5897-4bf4-8fa6-791913eac2ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:49.945018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527ddf4-5897-4bf4-8fa6-791913eac2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:49.945985 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527ddf4-5897-4bf4-8fa6-791913eac2ad admin admin] PATH_INFO: `/v3/users/b8774b57af854d47912a4468aaf94677` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:49.956187 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 899/1748] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:49 2022] PUT /identity/v3/projects/2c447b607152422aa828c2689eb0b824/users/e8d596b8e5f445ab8dc8bee1ea5fcb00/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:49.958398 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1527ddf4-5897-4bf4-8fa6-791913eac2ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8774b57af854d47912a4468aaf94677)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:49.960947 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15de5c6-a62e-4070-8921-a5d099386ee3 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:49.965331 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1527ddf4-5897-4bf4-8fa6-791913eac2ad admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:50.020575 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:50.063114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1527ddf4-5897-4bf4-8fa6-791913eac2ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8774b57af854d47912a4468aaf94677'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:50.074844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1527ddf4-5897-4bf4-8fa6-791913eac2ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8774b57af854d47912a4468aaf94677'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:50.091422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1527ddf4-5897-4bf4-8fa6-791913eac2ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8774b57af854d47912a4468aaf94677'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:50.091680 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15de5c6-a62e-4070-8921-a5d099386ee3 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:50.104702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a8efe-175f-4b8f-9cbe-0b0d9f00e746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:50.105760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a8efe-175f-4b8f-9cbe-0b0d9f00e746 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:50.105760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a8efe-175f-4b8f-9cbe-0b0d9f00e746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:50.105958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a8efe-175f-4b8f-9cbe-0b0d9f00e746 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:50.106753 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a8efe-175f-4b8f-9cbe-0b0d9f00e746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:50.108402 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a8efe-175f-4b8f-9cbe-0b0d9f00e746 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:50.124055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1527ddf4-5897-4bf4-8fa6-791913eac2ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8774b57af854d47912a4468aaf94677'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:50.129229 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1527ddf4-5897-4bf4-8fa6-791913eac2ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8774b57af854d47912a4468aaf94677'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:50.129841 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 900/1749] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:50.131035 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a8efe-175f-4b8f-9cbe-0b0d9f00e746 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:50.133626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8a7a8efe-175f-4b8f-9cbe-0b0d9f00e746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:50.135609 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 850/1750] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:38:49 2022] DELETE /identity/v3/users/b8774b57af854d47912a4468aaf94677 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:50.139794 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8212b-c36c-4b5f-9c56-95e876ca3ba3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:50.140088 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8212b-c36c-4b5f-9c56-95e876ca3ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:50.140370 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8212b-c36c-4b5f-9c56-95e876ca3ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:50.147754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a8efe-175f-4b8f-9cbe-0b0d9f00e746 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:50.158145 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a6f8212b-c36c-4b5f-9c56-95e876ca3ba3 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:50.159271 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a6f8212b-c36c-4b5f-9c56-95e876ca3ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:50.237282 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 851/1751] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:50.242527 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdbc10-6306-44a5-bec3-491b5f0527af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:50.243066 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8212b-c36c-4b5f-9c56-95e876ca3ba3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:50.243668 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdbc10-6306-44a5-bec3-491b5f0527af admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:50.243961 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdbc10-6306-44a5-bec3-491b5f0527af admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:50.244239 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdbc10-6306-44a5-bec3-491b5f0527af admin admin] PATH_INFO: `/v3/projects/2c447b607152422aa828c2689eb0b824/users/e8d596b8e5f445ab8dc8bee1ea5fcb00/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:50.246378 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdbc10-6306-44a5-bec3-491b5f0527af admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c447b607152422aa828c2689eb0b824, user_id=e8d596b8e5f445ab8dc8bee1ea5fcb00, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:50.248466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a6f8212b-c36c-4b5f-9c56-95e876ca3ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:50.249136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdbc10-6306-44a5-bec3-491b5f0527af admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:50.261054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 901/1752] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:50 2022] PUT /identity/v3/projects/2c447b607152422aa828c2689eb0b824/users/e8d596b8e5f445ab8dc8bee1ea5fcb00/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:38:50.272935 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eeb6fa-d9c7-4336-97b2-b5d98056abac None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:50.273713 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eeb6fa-d9c7-4336-97b2-b5d98056abac None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:50.274552 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eeb6fa-d9c7-4336-97b2-b5d98056abac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:50.311473 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-e1eeb6fa-d9c7-4336-97b2-b5d98056abac None None] MFA Rules not processed for user `e8d596b8e5f445ab8dc8bee1ea5fcb00`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:50.313799 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e1eeb6fa-d9c7-4336-97b2-b5d98056abac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:50.342031 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8212b-c36c-4b5f-9c56-95e876ca3ba3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:50.358159 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:50.419240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 902/1753] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:38:50.427415 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770c6b1-88b2-4d78-8bc8-db1cb40ff5c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:50.428092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770c6b1-88b2-4d78-8bc8-db1cb40ff5c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:50.428383 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770c6b1-88b2-4d78-8bc8-db1cb40ff5c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:50.428665 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770c6b1-88b2-4d78-8bc8-db1cb40ff5c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:50.433081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eeb6fa-d9c7-4336-97b2-b5d98056abac None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:50.434580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e1eeb6fa-d9c7-4336-97b2-b5d98056abac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:50.519233 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:50.544831 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eeb6fa-d9c7-4336-97b2-b5d98056abac None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:50.550904 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685ad99-4d50-46e5-8a70-c23b2c4344d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:50.552559 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685ad99-4d50-46e5-8a70-c23b2c4344d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:50.552973 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685ad99-4d50-46e5-8a70-c23b2c4344d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:50.553360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685ad99-4d50-46e5-8a70-c23b2c4344d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:50.568248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685ad99-4d50-46e5-8a70-c23b2c4344d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:50.571096 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685ad99-4d50-46e5-8a70-c23b2c4344d4 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:50.587845 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770c6b1-88b2-4d78-8bc8-db1cb40ff5c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:50.588026 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 903/1754] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:50.589568 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770c6b1-88b2-4d78-8bc8-db1cb40ff5c9 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:50.590355 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3509b9-4d2a-44fe-87a7-aa4f63dce4ea None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:50.590795 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3509b9-4d2a-44fe-87a7-aa4f63dce4ea None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:50.591510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3509b9-4d2a-44fe-87a7-aa4f63dce4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:50.620621 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 852/1755] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:38:50.625297 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-db3509b9-4d2a-44fe-87a7-aa4f63dce4ea None None] MFA Rules not processed for user `0b7d100f2407432381d119c25814fff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:50.627375 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-db3509b9-4d2a-44fe-87a7-aa4f63dce4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:50.735995 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 904/1756] 10.222.0.67 () {58 vars in 1009 bytes} [Tue May 31 14:38:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:50.764970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770c6b1-88b2-4d78-8bc8-db1cb40ff5c9 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:50.782545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770c6b1-88b2-4d78-8bc8-db1cb40ff5c9 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:50.802165 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77e313-2199-414a-aa1f-bc77332f5f2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:50.803485 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77e313-2199-414a-aa1f-bc77332f5f2d service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:50.803865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77e313-2199-414a-aa1f-bc77332f5f2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:50.806922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77e313-2199-414a-aa1f-bc77332f5f2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:50.812079 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-db77e313-2199-414a-aa1f-bc77332f5f2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:51.014829 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db77e313-2199-414a-aa1f-bc77332f5f2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:51.017606 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db77e313-2199-414a-aa1f-bc77332f5f2d service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:51.046900 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 853/1757] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:51.054042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3509b9-4d2a-44fe-87a7-aa4f63dce4ea None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:51.055792 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-db3509b9-4d2a-44fe-87a7-aa4f63dce4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:51.256814 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3509b9-4d2a-44fe-87a7-aa4f63dce4ea None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:51.269939 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da02f1f-28c0-4216-bd9a-ba49f721e9a6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfa2282f957941b98db1d7497da3351b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:51.271658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da02f1f-28c0-4216-bd9a-ba49f721e9a6 service swift] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:51.273771 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da02f1f-28c0-4216-bd9a-ba49f721e9a6 service swift] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:51.274062 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da02f1f-28c0-4216-bd9a-ba49f721e9a6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:51.301427 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da02f1f-28c0-4216-bd9a-ba49f721e9a6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:51.306109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da02f1f-28c0-4216-bd9a-ba49f721e9a6 service swift] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:51.328340 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 905/1758] 10.222.0.67 () {60 vars in 1384 bytes} [Tue May 31 14:38:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 31 14:38:53.427025 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77e313-2199-414a-aa1f-bc77332f5f2d service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:53.430219 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-db77e313-2199-414a-aa1f-bc77332f5f2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:53.496865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77e313-2199-414a-aa1f-bc77332f5f2d service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:53.536519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a451ff1-2436-43af-aca1-fa91c945d7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:53.537619 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a451ff1-2436-43af-aca1-fa91c945d7d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:53.541703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a451ff1-2436-43af-aca1-fa91c945d7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:53.542881 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a451ff1-2436-43af-aca1-fa91c945d7d8 admin admin] PATH_INFO: `/v3/projects/97bb9f0a317d46dbbbca44298eda83e4` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:53.551415 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a451ff1-2436-43af-aca1-fa91c945d7d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97bb9f0a317d46dbbbca44298eda83e4)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:53.560274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a451ff1-2436-43af-aca1-fa91c945d7d8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:53.751589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8a451ff1-2436-43af-aca1-fa91c945d7d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97bb9f0a317d46dbbbca44298eda83e4', 'request_id': 'req-8a451ff1-2436-43af-aca1-fa91c945d7d8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:53.784680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8a451ff1-2436-43af-aca1-fa91c945d7d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97bb9f0a317d46dbbbca44298eda83e4', 'request_id': 'req-8a451ff1-2436-43af-aca1-fa91c945d7d8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:53.795066 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:53.816218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8a451ff1-2436-43af-aca1-fa91c945d7d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97bb9f0a317d46dbbbca44298eda83e4', 'request_id': 'req-8a451ff1-2436-43af-aca1-fa91c945d7d8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:53.876251 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8a451ff1-2436-43af-aca1-fa91c945d7d8 admin admin] The token cache is being invalidate because project 97bb9f0a317d46dbbbca44298eda83e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:38:53.878007 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8a451ff1-2436-43af-aca1-fa91c945d7d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:53.878753 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8a451ff1-2436-43af-aca1-fa91c945d7d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:53.886373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 854/1759] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:38:53 2022] DELETE /identity/v3/projects/97bb9f0a317d46dbbbca44298eda83e4 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:53.893323 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da02f1f-28c0-4216-bd9a-ba49f721e9a6 service swift] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:53.920317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a451ff1-2436-43af-aca1-fa91c945d7d8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:53.922053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8a451ff1-2436-43af-aca1-fa91c945d7d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:53.930243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da02f1f-28c0-4216-bd9a-ba49f721e9a6 service swift] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:54.172122 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a451ff1-2436-43af-aca1-fa91c945d7d8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:54.189081 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732290e-d019-4b62-ab4a-9b3abf8f7509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:54.192035 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732290e-d019-4b62-ab4a-9b3abf8f7509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:54.194105 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732290e-d019-4b62-ab4a-9b3abf8f7509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:54.194483 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732290e-d019-4b62-ab4a-9b3abf8f7509 admin admin] PATH_INFO: `/v3/users/b7259f3236b3440a8826eeae2d5f3458` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:54.199492 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732290e-d019-4b62-ab4a-9b3abf8f7509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7259f3236b3440a8826eeae2d5f3458)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:54.208665 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732290e-d019-4b62-ab4a-9b3abf8f7509 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:54.214298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad975f-0723-4dce-9b18-d259ca261b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:54.219682 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad975f-0723-4dce-9b18-d259ca261b34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:54.222640 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad975f-0723-4dce-9b18-d259ca261b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:54.222640 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad975f-0723-4dce-9b18-d259ca261b34 admin admin] PATH_INFO: `/v3/users/0c843397485f4facbf3fbd98134f3eb9` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:54.230553 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad975f-0723-4dce-9b18-d259ca261b34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c843397485f4facbf3fbd98134f3eb9)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:54.234259 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad975f-0723-4dce-9b18-d259ca261b34 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:54.309654 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:54.397457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b732290e-d019-4b62-ab4a-9b3abf8f7509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7259f3236b3440a8826eeae2d5f3458'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:54.431895 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b732290e-d019-4b62-ab4a-9b3abf8f7509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7259f3236b3440a8826eeae2d5f3458'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:54.441840 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-52ad975f-0723-4dce-9b18-d259ca261b34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c843397485f4facbf3fbd98134f3eb9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:54.446208 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b732290e-d019-4b62-ab4a-9b3abf8f7509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7259f3236b3440a8826eeae2d5f3458'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:54.466481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-52ad975f-0723-4dce-9b18-d259ca261b34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c843397485f4facbf3fbd98134f3eb9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:54.467435 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-52ad975f-0723-4dce-9b18-d259ca261b34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c843397485f4facbf3fbd98134f3eb9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:54.483215 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b732290e-d019-4b62-ab4a-9b3abf8f7509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7259f3236b3440a8826eeae2d5f3458'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:54.488659 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b732290e-d019-4b62-ab4a-9b3abf8f7509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7259f3236b3440a8826eeae2d5f3458'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:54.498807 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-52ad975f-0723-4dce-9b18-d259ca261b34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c843397485f4facbf3fbd98134f3eb9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:54.501563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-52ad975f-0723-4dce-9b18-d259ca261b34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c843397485f4facbf3fbd98134f3eb9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:54.515636 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 855/1760] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:38:53 2022] DELETE /identity/v3/users/b7259f3236b3440a8826eeae2d5f3458 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:38:54.520884 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f412dc0-6ca0-4ec9-8b0a-7bf13f6abdf7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:54.524502 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f412dc0-6ca0-4ec9-8b0a-7bf13f6abdf7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:54.524973 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f412dc0-6ca0-4ec9-8b0a-7bf13f6abdf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:54.557530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 906/1761] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:38:53 2022] DELETE /identity/v3/users/0c843397485f4facbf3fbd98134f3eb9 => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:54.603652 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4f412dc0-6ca0-4ec9-8b0a-7bf13f6abdf7 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:54.604937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4f412dc0-6ca0-4ec9-8b0a-7bf13f6abdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:54.833519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 856/1762] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:54.878179 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef419ea9-64e8-4e11-883a-e1229fbd7b58 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:54.878507 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef419ea9-64e8-4e11-883a-e1229fbd7b58 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:54.878948 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef419ea9-64e8-4e11-883a-e1229fbd7b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:54.887192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f412dc0-6ca0-4ec9-8b0a-7bf13f6abdf7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:54.892314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4f412dc0-6ca0-4ec9-8b0a-7bf13f6abdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:54.917672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:54.979387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-ef419ea9-64e8-4e11-883a-e1229fbd7b58 None None] MFA Rules not processed for user `8966bc2ea7ff497c84214c9c9e6d53a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:54.985852 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ef419ea9-64e8-4e11-883a-e1229fbd7b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:55.207162 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f412dc0-6ca0-4ec9-8b0a-7bf13f6abdf7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:55.216771 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372e425-f1a7-404e-b710-95b65a35c745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:55.218087 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372e425-f1a7-404e-b710-95b65a35c745 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:55.218980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372e425-f1a7-404e-b710-95b65a35c745 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:55.219274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372e425-f1a7-404e-b710-95b65a35c745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:55.235126 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372e425-f1a7-404e-b710-95b65a35c745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:55.238094 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372e425-f1a7-404e-b710-95b65a35c745 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:55.284449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 857/1763] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:55.285616 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372e425-f1a7-404e-b710-95b65a35c745 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:55.295662 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b372e425-f1a7-404e-b710-95b65a35c745 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:55.315961 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 907/1764] 10.222.0.79 () {58 vars in 1015 bytes} [Tue May 31 14:38:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:55.355372 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372e425-f1a7-404e-b710-95b65a35c745 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:55.386919 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac9efb-b214-43d6-8ec0-31b1301a2507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:55.388824 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac9efb-b214-43d6-8ec0-31b1301a2507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:55.389533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac9efb-b214-43d6-8ec0-31b1301a2507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:55.390058 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac9efb-b214-43d6-8ec0-31b1301a2507 admin admin] PATH_INFO: `/v3/users/07826c2e4709491eb92a53473e02de73` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:55.392049 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac9efb-b214-43d6-8ec0-31b1301a2507 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07826c2e4709491eb92a53473e02de73)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:55.404053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac9efb-b214-43d6-8ec0-31b1301a2507 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:55.525300 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef419ea9-64e8-4e11-883a-e1229fbd7b58 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:55.537249 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef419ea9-64e8-4e11-883a-e1229fbd7b58 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:55.546357 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c2ac9efb-b214-43d6-8ec0-31b1301a2507 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07826c2e4709491eb92a53473e02de73'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:55.548370 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af2037-c1e7-4865-9e1b-e4b69c0b4d5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:55.549065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af2037-c1e7-4865-9e1b-e4b69c0b4d5e service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:55.549353 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af2037-c1e7-4865-9e1b-e4b69c0b4d5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:55.549703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af2037-c1e7-4865-9e1b-e4b69c0b4d5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:55.567079 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c2ac9efb-b214-43d6-8ec0-31b1301a2507 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07826c2e4709491eb92a53473e02de73'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:55.582227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af2037-c1e7-4865-9e1b-e4b69c0b4d5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:55.584046 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af2037-c1e7-4865-9e1b-e4b69c0b4d5e service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:55.592201 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c2ac9efb-b214-43d6-8ec0-31b1301a2507 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07826c2e4709491eb92a53473e02de73'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:55.614624 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 908/1765] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:38:55.627844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c2ac9efb-b214-43d6-8ec0-31b1301a2507 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07826c2e4709491eb92a53473e02de73'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:55.628720 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c2ac9efb-b214-43d6-8ec0-31b1301a2507 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07826c2e4709491eb92a53473e02de73'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:55.642501 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 858/1766] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:38:55 2022] DELETE /identity/v3/users/07826c2e4709491eb92a53473e02de73 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:55.646783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15342d9-b568-4894-8644-38fe828c87bc None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:55.647448 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15342d9-b568-4894-8644-38fe828c87bc None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:55.647448 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15342d9-b568-4894-8644-38fe828c87bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:55.668389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-b15342d9-b568-4894-8644-38fe828c87bc None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:55.670266 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b15342d9-b568-4894-8644-38fe828c87bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:55.765375 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 909/1767] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:55.815624 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac9efb-b214-43d6-8ec0-31b1301a2507 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:55.818783 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c2ac9efb-b214-43d6-8ec0-31b1301a2507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:56.105380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac9efb-b214-43d6-8ec0-31b1301a2507 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:56.114179 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfeb2f-40d3-4869-a74b-b1971e12e62b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:56.114965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfeb2f-40d3-4869-a74b-b1971e12e62b service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:56.115258 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfeb2f-40d3-4869-a74b-b1971e12e62b service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:56.115576 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfeb2f-40d3-4869-a74b-b1971e12e62b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:56.163059 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdfeb2f-40d3-4869-a74b-b1971e12e62b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:56.164892 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdfeb2f-40d3-4869-a74b-b1971e12e62b service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:56.203733 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 859/1768] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:56.245788 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15342d9-b568-4894-8644-38fe828c87bc None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:56.247281 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b15342d9-b568-4894-8644-38fe828c87bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:56.267128 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15342d9-b568-4894-8644-38fe828c87bc None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:56.307420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1564268-1de7-4da1-8867-4044596925c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:56.310095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1564268-1de7-4da1-8867-4044596925c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:56.311282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1564268-1de7-4da1-8867-4044596925c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:56.312457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1564268-1de7-4da1-8867-4044596925c1 admin admin] PATH_INFO: `/v3/projects/51ee47b55853470f859c88fa2fb01ee3` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:56.319284 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1564268-1de7-4da1-8867-4044596925c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51ee47b55853470f859c88fa2fb01ee3)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:56.325052 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1564268-1de7-4da1-8867-4044596925c1 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:56.475893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:56.531514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfeb2f-40d3-4869-a74b-b1971e12e62b service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:56.533372 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ccdfeb2f-40d3-4869-a74b-b1971e12e62b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:56.544110 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c1564268-1de7-4da1-8867-4044596925c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51ee47b55853470f859c88fa2fb01ee3', 'request_id': 'req-c1564268-1de7-4da1-8867-4044596925c1'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:56.550552 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfeb2f-40d3-4869-a74b-b1971e12e62b service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:56.556419 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c1564268-1de7-4da1-8867-4044596925c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51ee47b55853470f859c88fa2fb01ee3', 'request_id': 'req-c1564268-1de7-4da1-8867-4044596925c1'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:56.571915 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c1564268-1de7-4da1-8867-4044596925c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51ee47b55853470f859c88fa2fb01ee3', 'request_id': 'req-c1564268-1de7-4da1-8867-4044596925c1'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:56.600074 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c1564268-1de7-4da1-8867-4044596925c1 admin admin] The token cache is being invalidate because project 51ee47b55853470f859c88fa2fb01ee3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:38:56.603032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c1564268-1de7-4da1-8867-4044596925c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:56.604631 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c1564268-1de7-4da1-8867-4044596925c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:56.608830 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 910/1769] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:38:56 2022] DELETE /identity/v3/projects/51ee47b55853470f859c88fa2fb01ee3 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:56.617472 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba90b45-5f4c-45dc-8d6b-1702af323790 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:56.617965 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba90b45-5f4c-45dc-8d6b-1702af323790 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:56.618321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba90b45-5f4c-45dc-8d6b-1702af323790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:56.623146 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:56.665630 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-0ba90b45-5f4c-45dc-8d6b-1702af323790 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:56.666039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0ba90b45-5f4c-45dc-8d6b-1702af323790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:56.736594 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38725ba3-bbf9-4ff9-a3d7-be71f60d7bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:56.737529 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38725ba3-bbf9-4ff9-a3d7-be71f60d7bdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:56.737820 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38725ba3-bbf9-4ff9-a3d7-be71f60d7bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:56.738139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38725ba3-bbf9-4ff9-a3d7-be71f60d7bdb admin admin] PATH_INFO: `/v3/users/7ca6ef34e5624f868540d89c2a650e7b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:56.739515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38725ba3-bbf9-4ff9-a3d7-be71f60d7bdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ca6ef34e5624f868540d89c2a650e7b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:56.741950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38725ba3-bbf9-4ff9-a3d7-be71f60d7bdb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:56.845144 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-38725ba3-bbf9-4ff9-a3d7-be71f60d7bdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ca6ef34e5624f868540d89c2a650e7b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:56.862909 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 911/1770] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:56.864892 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-38725ba3-bbf9-4ff9-a3d7-be71f60d7bdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ca6ef34e5624f868540d89c2a650e7b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:56.874319 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3f77b-8f3e-47b8-bfaf-d08398e6f75f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:56.875708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3f77b-8f3e-47b8-bfaf-d08398e6f75f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:56.876755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3f77b-8f3e-47b8-bfaf-d08398e6f75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:56.886020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-38725ba3-bbf9-4ff9-a3d7-be71f60d7bdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ca6ef34e5624f868540d89c2a650e7b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:56.918664 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-38725ba3-bbf9-4ff9-a3d7-be71f60d7bdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ca6ef34e5624f868540d89c2a650e7b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:56.923161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-38725ba3-bbf9-4ff9-a3d7-be71f60d7bdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ca6ef34e5624f868540d89c2a650e7b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:56.927808 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-8ba3f77b-8f3e-47b8-bfaf-d08398e6f75f None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:56.929301 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8ba3f77b-8f3e-47b8-bfaf-d08398e6f75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:56.944711 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 860/1771] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:38:56 2022] DELETE /identity/v3/users/7ca6ef34e5624f868540d89c2a650e7b => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:56.947721 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95cd38-2f03-4a15-bf5b-ef336b4fbf8b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:56.948307 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95cd38-2f03-4a15-bf5b-ef336b4fbf8b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:56.950321 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95cd38-2f03-4a15-bf5b-ef336b4fbf8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:56.963764 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 912/1772] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:56.965670 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3f77b-8f3e-47b8-bfaf-d08398e6f75f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:56.985171 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ab95cd38-2f03-4a15-bf5b-ef336b4fbf8b None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:56.988434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ab95cd38-2f03-4a15-bf5b-ef336b4fbf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:56.997644 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3f77b-8f3e-47b8-bfaf-d08398e6f75f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:57.173149 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 861/1773] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:57.174238 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b16e5-08ae-4c7c-9b12-8c8609fd4063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:57.175219 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b16e5-08ae-4c7c-9b12-8c8609fd4063 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:57.175680 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b16e5-08ae-4c7c-9b12-8c8609fd4063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:57.176813 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b16e5-08ae-4c7c-9b12-8c8609fd4063 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:57.177306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b16e5-08ae-4c7c-9b12-8c8609fd4063 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:38:57.178881 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b16e5-08ae-4c7c-9b12-8c8609fd4063 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:57.180759 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b16e5-08ae-4c7c-9b12-8c8609fd4063 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:57.214165 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 913/1774] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:38:57.219919 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95cd38-2f03-4a15-bf5b-ef336b4fbf8b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:57.236597 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95cd38-2f03-4a15-bf5b-ef336b4fbf8b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:57.247244 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b16e5-08ae-4c7c-9b12-8c8609fd4063 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:57.249771 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-519b16e5-08ae-4c7c-9b12-8c8609fd4063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:57.261695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9399cdd-342f-4dc6-bcd6-eaf18a04dbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:57.262293 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9399cdd-342f-4dc6-bcd6-eaf18a04dbae admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:57.262618 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9399cdd-342f-4dc6-bcd6-eaf18a04dbae admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:57.262877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9399cdd-342f-4dc6-bcd6-eaf18a04dbae admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:57.263582 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9399cdd-342f-4dc6-bcd6-eaf18a04dbae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:57.288081 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9399cdd-342f-4dc6-bcd6-eaf18a04dbae admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:57.311562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:57.343092 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b16e5-08ae-4c7c-9b12-8c8609fd4063 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:57.345687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 862/1775] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:38:57 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:57.349277 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9399cdd-342f-4dc6-bcd6-eaf18a04dbae admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:57.417462 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:57.452100 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9a117-7db2-4b01-9a50-9bdf9709b954 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:57.452845 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9a117-7db2-4b01-9a50-9bdf9709b954 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:57.453198 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9a117-7db2-4b01-9a50-9bdf9709b954 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:57.454058 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9a117-7db2-4b01-9a50-9bdf9709b954 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:57.525078 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9399cdd-342f-4dc6-bcd6-eaf18a04dbae admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:57.540100 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ae42d-8c21-4cbe-bb17-65bbc5f22b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:57.540692 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ae42d-8c21-4cbe-bb17-65bbc5f22b01 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:57.540980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ae42d-8c21-4cbe-bb17-65bbc5f22b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:57.541388 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ae42d-8c21-4cbe-bb17-65bbc5f22b01 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:57.541958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ae42d-8c21-4cbe-bb17-65bbc5f22b01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:57.543928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ae42d-8c21-4cbe-bb17-65bbc5f22b01 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:57.567289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9a117-7db2-4b01-9a50-9bdf9709b954 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:57.569798 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9a117-7db2-4b01-9a50-9bdf9709b954 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:57.583446 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 863/1776] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:38:57 2022] POST /identity/v3/users => generated 290 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:57.587139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ae42d-8c21-4cbe-bb17-65bbc5f22b01 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:57.589958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 914/1777] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:38:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:38:57.605622 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ae42d-8c21-4cbe-bb17-65bbc5f22b01 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:57.619936 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed10012-2265-46f2-8dc1-f3f6c389ef7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:57.620649 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed10012-2265-46f2-8dc1-f3f6c389ef7f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:57.620923 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed10012-2265-46f2-8dc1-f3f6c389ef7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:57.621210 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed10012-2265-46f2-8dc1-f3f6c389ef7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:57.621953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed10012-2265-46f2-8dc1-f3f6c389ef7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:57.623645 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed10012-2265-46f2-8dc1-f3f6c389ef7f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:57.649074 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9a117-7db2-4b01-9a50-9bdf9709b954 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:57.655298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 864/1778] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:57.674441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9a117-7db2-4b01-9a50-9bdf9709b954 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:57.686832 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196d156-c128-4e16-9e2a-2c773aebe0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:57.688772 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196d156-c128-4e16-9e2a-2c773aebe0bd admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:57.689548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196d156-c128-4e16-9e2a-2c773aebe0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:57.690272 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196d156-c128-4e16-9e2a-2c773aebe0bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:57.695701 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196d156-c128-4e16-9e2a-2c773aebe0bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:57.731907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196d156-c128-4e16-9e2a-2c773aebe0bd admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:57.794093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 915/1779] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:38:57.798166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed10012-2265-46f2-8dc1-f3f6c389ef7f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:57.820226 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed10012-2265-46f2-8dc1-f3f6c389ef7f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:57.825071 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee415023-3146-4718-aec0-1cc92aca1c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:57.825992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee415023-3146-4718-aec0-1cc92aca1c49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:57.826283 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee415023-3146-4718-aec0-1cc92aca1c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:57.826569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee415023-3146-4718-aec0-1cc92aca1c49 admin admin] PATH_INFO: `/v3/projects/12aeb512b98a4ac0acf1ea520b404122/users/0cd817ab2cab4ebeac149395f8fec5ec/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:57.852199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee415023-3146-4718-aec0-1cc92aca1c49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12aeb512b98a4ac0acf1ea520b404122, user_id=0cd817ab2cab4ebeac149395f8fec5ec, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:57.854908 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee415023-3146-4718-aec0-1cc92aca1c49 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:57.872994 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 865/1780] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:57 2022] PUT /identity/v3/projects/12aeb512b98a4ac0acf1ea520b404122/users/0cd817ab2cab4ebeac149395f8fec5ec/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:57.883870 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749f8f6-2795-4fca-bc90-fe6d7a3778d6 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:57.884179 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749f8f6-2795-4fca-bc90-fe6d7a3778d6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:57.884504 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749f8f6-2795-4fca-bc90-fe6d7a3778d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:57.975989 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-4749f8f6-2795-4fca-bc90-fe6d7a3778d6 None None] MFA Rules not processed for user `0cd817ab2cab4ebeac149395f8fec5ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:57.978095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4749f8f6-2795-4fca-bc90-fe6d7a3778d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:58.149440 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 916/1781] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:38:58.151099 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee415023-3146-4718-aec0-1cc92aca1c49 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:58.329241 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee415023-3146-4718-aec0-1cc92aca1c49 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:58.348400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93231eb-46cd-44f3-9fc8-794937b0dce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:58.351042 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93231eb-46cd-44f3-9fc8-794937b0dce2 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:58.351344 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93231eb-46cd-44f3-9fc8-794937b0dce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:58.352188 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93231eb-46cd-44f3-9fc8-794937b0dce2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:58.356737 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93231eb-46cd-44f3-9fc8-794937b0dce2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:58.362505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93231eb-46cd-44f3-9fc8-794937b0dce2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:58.428733 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 866/1782] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:38:58 2022] POST /identity/v3/projects => generated 377 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:38:58.431649 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749f8f6-2795-4fca-bc90-fe6d7a3778d6 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:58.502126 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749f8f6-2795-4fca-bc90-fe6d7a3778d6 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:58.688639 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:58.839121 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730260b5-3cb7-4b4b-9de5-4277c80efaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:58.840037 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730260b5-3cb7-4b4b-9de5-4277c80efaf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:58.843604 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730260b5-3cb7-4b4b-9de5-4277c80efaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:58.844011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730260b5-3cb7-4b4b-9de5-4277c80efaf5 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:58.845430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730260b5-3cb7-4b4b-9de5-4277c80efaf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:58.856528 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730260b5-3cb7-4b4b-9de5-4277c80efaf5 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:58.914934 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93231eb-46cd-44f3-9fc8-794937b0dce2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:58.915526 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 917/1783] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:38:58 2022] POST /identity/v3/users => generated 289 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:58.928978 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93231eb-46cd-44f3-9fc8-794937b0dce2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:58.949919 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14bfd8b-cb44-4dfe-a276-dbe8f5489720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:58.950876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14bfd8b-cb44-4dfe-a276-dbe8f5489720 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:58.951386 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14bfd8b-cb44-4dfe-a276-dbe8f5489720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:58.958331 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14bfd8b-cb44-4dfe-a276-dbe8f5489720 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:58.960361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14bfd8b-cb44-4dfe-a276-dbe8f5489720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:58.964019 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14bfd8b-cb44-4dfe-a276-dbe8f5489720 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:58.983642 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 867/1784] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:58.984568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730260b5-3cb7-4b4b-9de5-4277c80efaf5 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:59.009514 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730260b5-3cb7-4b4b-9de5-4277c80efaf5 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:59.020252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c37aa-b198-4142-b176-9303efea485f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:59.021401 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c37aa-b198-4142-b176-9303efea485f admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:59.021846 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c37aa-b198-4142-b176-9303efea485f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:59.022483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c37aa-b198-4142-b176-9303efea485f admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:59.023491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c37aa-b198-4142-b176-9303efea485f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:59.028198 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c37aa-b198-4142-b176-9303efea485f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:59.081609 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14bfd8b-cb44-4dfe-a276-dbe8f5489720 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:59.091068 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 918/1785] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:38:59.149756 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14bfd8b-cb44-4dfe-a276-dbe8f5489720 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:59.176534 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b3028-82bc-4869-b61c-b042e8ad67d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:59.177960 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b3028-82bc-4869-b61c-b042e8ad67d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:59.178324 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b3028-82bc-4869-b61c-b042e8ad67d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:59.178841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b3028-82bc-4869-b61c-b042e8ad67d0 admin admin] PATH_INFO: `/v3/projects/82cb9f1f05af4e50805cce3aff7964a1/users/0d9fa91ba9084d0a830c54a1d8b56aa9/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:59.218525 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b3028-82bc-4869-b61c-b042e8ad67d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82cb9f1f05af4e50805cce3aff7964a1, user_id=0d9fa91ba9084d0a830c54a1d8b56aa9, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:59.229413 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b3028-82bc-4869-b61c-b042e8ad67d0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:59.284940 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 868/1786] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:38:59 2022] PUT /identity/v3/projects/82cb9f1f05af4e50805cce3aff7964a1/users/0d9fa91ba9084d0a830c54a1d8b56aa9/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:59.285840 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92244ef5-7e7a-4794-823f-3b912f9e4841 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:59.286262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92244ef5-7e7a-4794-823f-3b912f9e4841 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:59.286562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92244ef5-7e7a-4794-823f-3b912f9e4841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:59.366249 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-92244ef5-7e7a-4794-823f-3b912f9e4841 None None] MFA Rules not processed for user `0d9fa91ba9084d0a830c54a1d8b56aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:38:59.371619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-92244ef5-7e7a-4794-823f-3b912f9e4841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:59.511408 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 919/1787] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:38:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:38:59.515356 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b3028-82bc-4869-b61c-b042e8ad67d0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:59.516247 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92244ef5-7e7a-4794-823f-3b912f9e4841 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:59.517421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-463b3028-82bc-4869-b61c-b042e8ad67d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:59.621314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:59.684218 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92244ef5-7e7a-4794-823f-3b912f9e4841 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:59.697566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4e018-3278-43aa-b53e-64b21064935f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:59.698270 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4e018-3278-43aa-b53e-64b21064935f admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:59.698561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4e018-3278-43aa-b53e-64b21064935f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:59.698873 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4e018-3278-43aa-b53e-64b21064935f admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:59.699600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f4e018-3278-43aa-b53e-64b21064935f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:59.701589 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f4e018-3278-43aa-b53e-64b21064935f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:59.703700 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b3028-82bc-4869-b61c-b042e8ad67d0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:38:59.712035 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d2996-1f50-4712-b555-93ddda888ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:38:59.713735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d2996-1f50-4712-b555-93ddda888ce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:38:59.714226 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d2996-1f50-4712-b555-93ddda888ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:38:59.714717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d2996-1f50-4712-b555-93ddda888ce2 admin admin] PATH_INFO: `/v3/projects/91cafdeef693436ebfd7ad38408fe5ba` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:38:59.716914 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d2996-1f50-4712-b555-93ddda888ce2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91cafdeef693436ebfd7ad38408fe5ba)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:38:59.719330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d2996-1f50-4712-b555-93ddda888ce2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:38:59.757194 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 920/1788] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:38:59 2022] POST /identity/v3/projects => generated 379 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:38:59.758913 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4e018-3278-43aa-b53e-64b21064935f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:59.811134 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6f1d2996-1f50-4712-b555-93ddda888ce2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91cafdeef693436ebfd7ad38408fe5ba', 'request_id': 'req-6f1d2996-1f50-4712-b555-93ddda888ce2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:59.826162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:38:59.828038 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6f1d2996-1f50-4712-b555-93ddda888ce2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91cafdeef693436ebfd7ad38408fe5ba', 'request_id': 'req-6f1d2996-1f50-4712-b555-93ddda888ce2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:59.853854 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6f1d2996-1f50-4712-b555-93ddda888ce2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91cafdeef693436ebfd7ad38408fe5ba', 'request_id': 'req-6f1d2996-1f50-4712-b555-93ddda888ce2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:59.899858 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6f1d2996-1f50-4712-b555-93ddda888ce2 admin admin] The token cache is being invalidate because project 91cafdeef693436ebfd7ad38408fe5ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:38:59.899858 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6f1d2996-1f50-4712-b555-93ddda888ce2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:59.899858 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6f1d2996-1f50-4712-b555-93ddda888ce2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:38:59.911205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 869/1789] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:38:59 2022] DELETE /identity/v3/projects/91cafdeef693436ebfd7ad38408fe5ba => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:38:59.915297 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d2996-1f50-4712-b555-93ddda888ce2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:38:59.922439 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6f1d2996-1f50-4712-b555-93ddda888ce2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:38:59.968695 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4e018-3278-43aa-b53e-64b21064935f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:00.063944 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d2996-1f50-4712-b555-93ddda888ce2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:00.078065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f9603-ba2d-4382-9e8d-ddc7960e89e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:00.078914 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f9603-ba2d-4382-9e8d-ddc7960e89e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:00.079308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f9603-ba2d-4382-9e8d-ddc7960e89e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:00.079492 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f9603-ba2d-4382-9e8d-ddc7960e89e7 admin admin] PATH_INFO: `/v3/users/f775c78c25584769b493376fe2d8fe0c` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:00.081616 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f9603-ba2d-4382-9e8d-ddc7960e89e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f775c78c25584769b493376fe2d8fe0c)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:00.083599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f9603-ba2d-4382-9e8d-ddc7960e89e7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:00.150807 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e756f-a317-4937-8eb1-8c2d15216761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:00.155729 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e756f-a317-4937-8eb1-8c2d15216761 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:00.156222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e756f-a317-4937-8eb1-8c2d15216761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:00.156694 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e756f-a317-4937-8eb1-8c2d15216761 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:00.157627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e756f-a317-4937-8eb1-8c2d15216761 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:00.162365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e756f-a317-4937-8eb1-8c2d15216761 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:00.208147 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-564f9603-ba2d-4382-9e8d-ddc7960e89e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f775c78c25584769b493376fe2d8fe0c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:00.217117 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 921/1790] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:38:59 2022] POST /identity/v3/users => generated 296 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:00.218730 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27240f6f-57b6-406e-b16b-c6aba3d0120b None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:00.220359 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27240f6f-57b6-406e-b16b-c6aba3d0120b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:00.222358 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27240f6f-57b6-406e-b16b-c6aba3d0120b None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:00.224139 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 922/1791] 10.222.0.67 () {54 vars in 921 bytes} [Tue May 31 14:39:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:39:00.231255 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-564f9603-ba2d-4382-9e8d-ddc7960e89e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f775c78c25584769b493376fe2d8fe0c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:00.233645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27240f6f-57b6-406e-b16b-c6aba3d0120b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:00.236542 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-27240f6f-57b6-406e-b16b-c6aba3d0120b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:00.256034 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:00.259196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-564f9603-ba2d-4382-9e8d-ddc7960e89e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f775c78c25584769b493376fe2d8fe0c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:00.283992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27240f6f-57b6-406e-b16b-c6aba3d0120b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:00.291418 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb9064-630e-406e-aefa-3e09e9a6d79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:00.292562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb9064-630e-406e-aefa-3e09e9a6d79e admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:00.292963 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb9064-630e-406e-aefa-3e09e9a6d79e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:00.293369 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb9064-630e-406e-aefa-3e09e9a6d79e admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:00.303991 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb9064-630e-406e-aefa-3e09e9a6d79e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:00.306408 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb9064-630e-406e-aefa-3e09e9a6d79e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:00.316952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-564f9603-ba2d-4382-9e8d-ddc7960e89e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f775c78c25584769b493376fe2d8fe0c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:00.317889 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-564f9603-ba2d-4382-9e8d-ddc7960e89e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f775c78c25584769b493376fe2d8fe0c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:00.335487 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 870/1792] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:38:59 2022] DELETE /identity/v3/users/f775c78c25584769b493376fe2d8fe0c => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:00.335487 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec817f2e-c2f5-4e37-9ff9-95d8b7bc10b8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:00.335487 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec817f2e-c2f5-4e37-9ff9-95d8b7bc10b8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:00.335487 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec817f2e-c2f5-4e37-9ff9-95d8b7bc10b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:00.339701 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 923/1793] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:00.341053 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb9064-630e-406e-aefa-3e09e9a6d79e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:00.350882 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c1fb9064-630e-406e-aefa-3e09e9a6d79e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:00.375657 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ec817f2e-c2f5-4e37-9ff9-95d8b7bc10b8 None None] MFA Rules not processed for user `68c6363b60954c3ebf50d2019f5f2fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:00.376957 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ec817f2e-c2f5-4e37-9ff9-95d8b7bc10b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:00.402117 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb9064-630e-406e-aefa-3e09e9a6d79e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:00.564198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 871/1794] 10.222.0.67 () {58 vars in 1018 bytes} [Tue May 31 14:39:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:00.568971 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da98dfe-cf43-4b65-8fc9-1e731dda9449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:00.570132 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da98dfe-cf43-4b65-8fc9-1e731dda9449 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:00.570513 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da98dfe-cf43-4b65-8fc9-1e731dda9449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:00.571178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da98dfe-cf43-4b65-8fc9-1e731dda9449 admin admin] PATH_INFO: `/v3/projects/ffcebece2ea54b3ba4bf5de3660aa03d/users/27e384d4c6954153b87481b3bfd8956e/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:00.597340 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da98dfe-cf43-4b65-8fc9-1e731dda9449 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffcebece2ea54b3ba4bf5de3660aa03d, user_id=27e384d4c6954153b87481b3bfd8956e, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:00.607595 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da98dfe-cf43-4b65-8fc9-1e731dda9449 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:00.690726 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec817f2e-c2f5-4e37-9ff9-95d8b7bc10b8 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:00.721494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 924/1795] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:00 2022] PUT /identity/v3/projects/ffcebece2ea54b3ba4bf5de3660aa03d/users/27e384d4c6954153b87481b3bfd8956e/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:00.722542 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da98dfe-cf43-4b65-8fc9-1e731dda9449 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:00.738585 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec817f2e-c2f5-4e37-9ff9-95d8b7bc10b8 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:00.738998 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6da98dfe-cf43-4b65-8fc9-1e731dda9449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:00.857674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:00.933317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0d2c3-fb37-462d-96ee-c3569f0ad547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:00.934624 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0d2c3-fb37-462d-96ee-c3569f0ad547 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:00.935144 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0d2c3-fb37-462d-96ee-c3569f0ad547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:00.935731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0d2c3-fb37-462d-96ee-c3569f0ad547 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:00.936689 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0d2c3-fb37-462d-96ee-c3569f0ad547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:00.938701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0d2c3-fb37-462d-96ee-c3569f0ad547 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:00.969376 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 872/1796] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:00.973161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0d2c3-fb37-462d-96ee-c3569f0ad547 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:01.003655 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0d2c3-fb37-462d-96ee-c3569f0ad547 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:01.019903 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be931c-ddc8-47bf-a4f7-9ac4623a5752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:01.019903 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be931c-ddc8-47bf-a4f7-9ac4623a5752 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:01.019903 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be931c-ddc8-47bf-a4f7-9ac4623a5752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:01.019903 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be931c-ddc8-47bf-a4f7-9ac4623a5752 admin admin] PATH_INFO: `/v3/projects/ffcebece2ea54b3ba4bf5de3660aa03d/users/27e384d4c6954153b87481b3bfd8956e/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:01.022261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be931c-ddc8-47bf-a4f7-9ac4623a5752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffcebece2ea54b3ba4bf5de3660aa03d, user_id=27e384d4c6954153b87481b3bfd8956e, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:01.027163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be931c-ddc8-47bf-a4f7-9ac4623a5752 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:01.031309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da98dfe-cf43-4b65-8fc9-1e731dda9449 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:01.046942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f51ac9-deb6-4840-ac41-8a2144723a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:01.047858 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f51ac9-deb6-4840-ac41-8a2144723a24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:01.050144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f51ac9-deb6-4840-ac41-8a2144723a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:01.050548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f51ac9-deb6-4840-ac41-8a2144723a24 admin admin] PATH_INFO: `/v3/projects/46aee1976e2542bfaaf81e1ee728a29d` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:01.054600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f51ac9-deb6-4840-ac41-8a2144723a24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46aee1976e2542bfaaf81e1ee728a29d)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:01.062804 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f51ac9-deb6-4840-ac41-8a2144723a24 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:01.065336 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 873/1797] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:00 2022] PUT /identity/v3/projects/ffcebece2ea54b3ba4bf5de3660aa03d/users/27e384d4c6954153b87481b3bfd8956e/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:01.077362 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ff709-b808-4e3d-a8e9-39a2ad1fade1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:01.077685 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ff709-b808-4e3d-a8e9-39a2ad1fade1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:01.077922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ff709-b808-4e3d-a8e9-39a2ad1fade1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:01.162118 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-aa2ff709-b808-4e3d-a8e9-39a2ad1fade1 None None] MFA Rules not processed for user `27e384d4c6954153b87481b3bfd8956e`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:01.163299 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-aa2ff709-b808-4e3d-a8e9-39a2ad1fade1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:01.234783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-64f51ac9-deb6-4840-ac41-8a2144723a24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46aee1976e2542bfaaf81e1ee728a29d', 'request_id': 'req-64f51ac9-deb6-4840-ac41-8a2144723a24'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:01.277036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-64f51ac9-deb6-4840-ac41-8a2144723a24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46aee1976e2542bfaaf81e1ee728a29d', 'request_id': 'req-64f51ac9-deb6-4840-ac41-8a2144723a24'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:01.298143 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:01.321024 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-64f51ac9-deb6-4840-ac41-8a2144723a24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46aee1976e2542bfaaf81e1ee728a29d', 'request_id': 'req-64f51ac9-deb6-4840-ac41-8a2144723a24'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:01.346479 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-64f51ac9-deb6-4840-ac41-8a2144723a24 admin admin] The token cache is being invalidate because project 46aee1976e2542bfaaf81e1ee728a29d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:01.346812 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-64f51ac9-deb6-4840-ac41-8a2144723a24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:01.347470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-64f51ac9-deb6-4840-ac41-8a2144723a24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:01.353371 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 925/1798] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:00 2022] DELETE /identity/v3/projects/46aee1976e2542bfaaf81e1ee728a29d => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:01.356045 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcf932-d12a-47b6-88b9-a7a799e7d1d2 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:01.356307 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcf932-d12a-47b6-88b9-a7a799e7d1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:01.356577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcf932-d12a-47b6-88b9-a7a799e7d1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:01.395570 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 874/1799] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:01 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:39:01.401378 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-7ddcf932-d12a-47b6-88b9-a7a799e7d1d2 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:01.402690 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7ddcf932-d12a-47b6-88b9-a7a799e7d1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:01.413511 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ff709-b808-4e3d-a8e9-39a2ad1fade1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:01.416167 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-aa2ff709-b808-4e3d-a8e9-39a2ad1fade1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:01.581946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ff709-b808-4e3d-a8e9-39a2ad1fade1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:01.593705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3d847-a0fa-48ff-b000-ed5732454a61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:01.595627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3d847-a0fa-48ff-b000-ed5732454a61 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:01.596488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3d847-a0fa-48ff-b000-ed5732454a61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:01.597705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3d847-a0fa-48ff-b000-ed5732454a61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:01.613235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 926/1800] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:01.621832 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddcf932-d12a-47b6-88b9-a7a799e7d1d2 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:01.623300 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7ddcf932-d12a-47b6-88b9-a7a799e7d1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:01.635643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e3d847-a0fa-48ff-b000-ed5732454a61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:01.637940 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e3d847-a0fa-48ff-b000-ed5732454a61 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:01.642065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddcf932-d12a-47b6-88b9-a7a799e7d1d2 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:01.658172 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584982d-21ad-4cbd-8b75-0f2ff2473425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:01.658878 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584982d-21ad-4cbd-8b75-0f2ff2473425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:01.659162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584982d-21ad-4cbd-8b75-0f2ff2473425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:01.659476 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584982d-21ad-4cbd-8b75-0f2ff2473425 admin admin] PATH_INFO: `/v3/projects/6a12403ef5d54834b76c69b1315174a3` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:01.660705 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6584982d-21ad-4cbd-8b75-0f2ff2473425 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a12403ef5d54834b76c69b1315174a3)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:01.662940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6584982d-21ad-4cbd-8b75-0f2ff2473425 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:01.681889 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 875/1801] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:01 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:39:01.689340 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd2a67-dcda-4e93-9f09-84d2cf27bb7f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:01.690684 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd2a67-dcda-4e93-9f09-84d2cf27bb7f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:01.691383 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd2a67-dcda-4e93-9f09-84d2cf27bb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:01.739648 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f2cd2a67-dcda-4e93-9f09-84d2cf27bb7f None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:01.741581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f2cd2a67-dcda-4e93-9f09-84d2cf27bb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:01.813639 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6584982d-21ad-4cbd-8b75-0f2ff2473425 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a12403ef5d54834b76c69b1315174a3', 'request_id': 'req-6584982d-21ad-4cbd-8b75-0f2ff2473425'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:01.834931 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:01.845581 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6584982d-21ad-4cbd-8b75-0f2ff2473425 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a12403ef5d54834b76c69b1315174a3', 'request_id': 'req-6584982d-21ad-4cbd-8b75-0f2ff2473425'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:01.884214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6584982d-21ad-4cbd-8b75-0f2ff2473425 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a12403ef5d54834b76c69b1315174a3', 'request_id': 'req-6584982d-21ad-4cbd-8b75-0f2ff2473425'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:01.905736 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6584982d-21ad-4cbd-8b75-0f2ff2473425 admin admin] The token cache is being invalidate because project 6a12403ef5d54834b76c69b1315174a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:01.906191 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6584982d-21ad-4cbd-8b75-0f2ff2473425 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:01.907103 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6584982d-21ad-4cbd-8b75-0f2ff2473425 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:01.913962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 927/1802] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:01 2022] DELETE /identity/v3/projects/6a12403ef5d54834b76c69b1315174a3 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:01.913962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584982d-21ad-4cbd-8b75-0f2ff2473425 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:01.916341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6584982d-21ad-4cbd-8b75-0f2ff2473425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:01.966774 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 876/1803] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:01.968856 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd2a67-dcda-4e93-9f09-84d2cf27bb7f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:01.972838 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f2cd2a67-dcda-4e93-9f09-84d2cf27bb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:02.093392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584982d-21ad-4cbd-8b75-0f2ff2473425 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:02.109785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712e981-91bd-4739-aca9-96a08653a40a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:02.110475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712e981-91bd-4739-aca9-96a08653a40a service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:02.110790 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712e981-91bd-4739-aca9-96a08653a40a service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:02.111075 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712e981-91bd-4739-aca9-96a08653a40a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:02.122024 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b712e981-91bd-4739-aca9-96a08653a40a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:02.264989 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd2a67-dcda-4e93-9f09-84d2cf27bb7f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:02.303598 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0aa7e2-11f8-4ab0-81d1-10530b939476 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:02.319575 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0aa7e2-11f8-4ab0-81d1-10530b939476 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:02.321462 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0aa7e2-11f8-4ab0-81d1-10530b939476 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:02.321978 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0aa7e2-11f8-4ab0-81d1-10530b939476 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:02.327265 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ed0aa7e2-11f8-4ab0-81d1-10530b939476 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:02.380853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0aa7e2-11f8-4ab0-81d1-10530b939476 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:02.389052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0aa7e2-11f8-4ab0-81d1-10530b939476 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:02.399512 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b712e981-91bd-4739-aca9-96a08653a40a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:02.403452 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b712e981-91bd-4739-aca9-96a08653a40a service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:02.437977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 877/1804] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:02.444744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0aa7e2-11f8-4ab0-81d1-10530b939476 service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:02.447955 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 928/1805] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:02.450558 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ed0aa7e2-11f8-4ab0-81d1-10530b939476 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:02.454172 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712e981-91bd-4739-aca9-96a08653a40a service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:02.501545 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712e981-91bd-4739-aca9-96a08653a40a service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:02.510327 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0aa7e2-11f8-4ab0-81d1-10530b939476 service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:02.527298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836321c-a97d-46e1-bff1-e9e513de30d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:02.528533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836321c-a97d-46e1-bff1-e9e513de30d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:02.529433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836321c-a97d-46e1-bff1-e9e513de30d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:02.529899 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836321c-a97d-46e1-bff1-e9e513de30d2 admin admin] PATH_INFO: `/v3/users/e8d596b8e5f445ab8dc8bee1ea5fcb00` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:02.531709 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26da19d-a703-4a85-82fb-dfbcd2ca4bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:02.532413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26da19d-a703-4a85-82fb-dfbcd2ca4bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:02.532702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26da19d-a703-4a85-82fb-dfbcd2ca4bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:02.533312 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9836321c-a97d-46e1-bff1-e9e513de30d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8d596b8e5f445ab8dc8bee1ea5fcb00)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:02.535973 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9836321c-a97d-46e1-bff1-e9e513de30d2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:02.536418 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26da19d-a703-4a85-82fb-dfbcd2ca4bf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:02.537220 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26da19d-a703-4a85-82fb-dfbcd2ca4bf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:39:02.537532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26da19d-a703-4a85-82fb-dfbcd2ca4bf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:02.538320 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:02.544540 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26da19d-a703-4a85-82fb-dfbcd2ca4bf6 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:02.600846 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 929/1806] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:39:02.602504 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26da19d-a703-4a85-82fb-dfbcd2ca4bf6 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:02.607623 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d26da19d-a703-4a85-82fb-dfbcd2ca4bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:02.663277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26da19d-a703-4a85-82fb-dfbcd2ca4bf6 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:02.684179 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ecc432-ea75-4155-8258-b51273099e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:02.684359 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecc432-ea75-4155-8258-b51273099e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:02.684595 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecc432-ea75-4155-8258-b51273099e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:02.684904 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecc432-ea75-4155-8258-b51273099e0d admin admin] PATH_INFO: `/v3/projects/039c26e374e249689d9251848bb98ec0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:02.688421 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ecc432-ea75-4155-8258-b51273099e0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=039c26e374e249689d9251848bb98ec0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:02.692376 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ecc432-ea75-4155-8258-b51273099e0d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:02.753582 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9836321c-a97d-46e1-bff1-e9e513de30d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8d596b8e5f445ab8dc8bee1ea5fcb00'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:02.774069 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9836321c-a97d-46e1-bff1-e9e513de30d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8d596b8e5f445ab8dc8bee1ea5fcb00'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:02.792393 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9836321c-a97d-46e1-bff1-e9e513de30d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8d596b8e5f445ab8dc8bee1ea5fcb00'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:02.817576 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9836321c-a97d-46e1-bff1-e9e513de30d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8d596b8e5f445ab8dc8bee1ea5fcb00'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:02.818768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9836321c-a97d-46e1-bff1-e9e513de30d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8d596b8e5f445ab8dc8bee1ea5fcb00'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:02.840417 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 878/1807] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:02 2022] DELETE /identity/v3/users/e8d596b8e5f445ab8dc8bee1ea5fcb00 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:02.843427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b4179-afab-4f68-8174-fca8b6b2f48a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:02.843427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b4179-afab-4f68-8174-fca8b6b2f48a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:02.843427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b4179-afab-4f68-8174-fca8b6b2f48a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:02.844998 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 879/1808] 10.222.0.67 () {54 vars in 921 bytes} [Tue May 31 14:39:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:39:02.847847 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b4179-afab-4f68-8174-fca8b6b2f48a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:02.856024 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fa6b4179-afab-4f68-8174-fca8b6b2f48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:02.893726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-31ecc432-ea75-4155-8258-b51273099e0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '039c26e374e249689d9251848bb98ec0', 'request_id': 'req-31ecc432-ea75-4155-8258-b51273099e0d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:02.917354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-31ecc432-ea75-4155-8258-b51273099e0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '039c26e374e249689d9251848bb98ec0', 'request_id': 'req-31ecc432-ea75-4155-8258-b51273099e0d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:02.936199 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-31ecc432-ea75-4155-8258-b51273099e0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '039c26e374e249689d9251848bb98ec0', 'request_id': 'req-31ecc432-ea75-4155-8258-b51273099e0d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:02.978725 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-31ecc432-ea75-4155-8258-b51273099e0d admin admin] The token cache is being invalidate because project 039c26e374e249689d9251848bb98ec0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:02.979059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-31ecc432-ea75-4155-8258-b51273099e0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:02.979999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-31ecc432-ea75-4155-8258-b51273099e0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:02.982364 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 930/1809] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:02 2022] DELETE /identity/v3/projects/039c26e374e249689d9251848bb98ec0 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:02.989436 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6febd5-f6d2-4450-9c50-2d5c7c739ae1 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:02.989724 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6febd5-f6d2-4450-9c50-2d5c7c739ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:02.990341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6febd5-f6d2-4450-9c50-2d5c7c739ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:03.018987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:03.040800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-ff6febd5-f6d2-4450-9c50-2d5c7c739ae1 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:03.045086 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ff6febd5-f6d2-4450-9c50-2d5c7c739ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:03.053760 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b4179-afab-4f68-8174-fca8b6b2f48a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:03.067331 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba5cd0-123a-4018-b5b7-666e3859b1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:03.071181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 931/1810] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:39:03.076072 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba5cd0-123a-4018-b5b7-666e3859b1a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:03.077002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba5cd0-123a-4018-b5b7-666e3859b1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:03.077226 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca85bc0-f32b-46e9-bad0-d9bf85cf840f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:03.077540 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca85bc0-f32b-46e9-bad0-d9bf85cf840f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:03.077861 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca85bc0-f32b-46e9-bad0-d9bf85cf840f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:03.080129 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba5cd0-123a-4018-b5b7-666e3859b1a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:03.081153 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba5cd0-123a-4018-b5b7-666e3859b1a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:03.095419 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba5cd0-123a-4018-b5b7-666e3859b1a9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:03.116284 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-eca85bc0-f32b-46e9-bad0-d9bf85cf840f None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:03.134596 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-eca85bc0-f32b-46e9-bad0-d9bf85cf840f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:03.151198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 880/1811] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:02 2022] POST /identity/v3/projects => generated 363 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:03.153253 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba04b7b-28f7-49dc-9bca-dd880eb56dab None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:03.155542 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba04b7b-28f7-49dc-9bca-dd880eb56dab None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:03.157037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba04b7b-28f7-49dc-9bca-dd880eb56dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:03.163069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 932/1812] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:03.163069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca85bc0-f32b-46e9-bad0-d9bf85cf840f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:03.164844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-eca85bc0-f32b-46e9-bad0-d9bf85cf840f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:03.189483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca85bc0-f32b-46e9-bad0-d9bf85cf840f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:03.204686 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1ba04b7b-28f7-49dc-9bca-dd880eb56dab None None] MFA Rules not processed for user `68c6363b60954c3ebf50d2019f5f2fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:03.206081 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1ba04b7b-28f7-49dc-9bca-dd880eb56dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:03.367261 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eacb1c1-98d6-42a8-a107-37a54c960f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:03.368115 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacb1c1-98d6-42a8-a107-37a54c960f56 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:03.368705 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacb1c1-98d6-42a8-a107-37a54c960f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:03.369064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacb1c1-98d6-42a8-a107-37a54c960f56 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:03.369887 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eacb1c1-98d6-42a8-a107-37a54c960f56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:03.374075 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eacb1c1-98d6-42a8-a107-37a54c960f56 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:03.400936 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:03.420866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 933/1813] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:03 2022] POST /identity/v3/users => generated 282 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:03.421812 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eacb1c1-98d6-42a8-a107-37a54c960f56 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:03.440257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 881/1814] 10.222.0.67 () {58 vars in 1018 bytes} [Tue May 31 14:39:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:03.440997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba04b7b-28f7-49dc-9bca-dd880eb56dab None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:03.443334 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eacb1c1-98d6-42a8-a107-37a54c960f56 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:03.459290 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c572203-a2de-4822-8cd0-a025fe8857a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:03.459986 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c572203-a2de-4822-8cd0-a025fe8857a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:03.460428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c572203-a2de-4822-8cd0-a025fe8857a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:03.461255 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c572203-a2de-4822-8cd0-a025fe8857a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:03.462169 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c572203-a2de-4822-8cd0-a025fe8857a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:39:03.462487 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c572203-a2de-4822-8cd0-a025fe8857a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:03.464297 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c572203-a2de-4822-8cd0-a025fe8857a7 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:03.465955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba04b7b-28f7-49dc-9bca-dd880eb56dab None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:03.474705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d47b1-f6c8-45a0-b0e4-609d80684107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:03.475878 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d47b1-f6c8-45a0-b0e4-609d80684107 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:03.476174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d47b1-f6c8-45a0-b0e4-609d80684107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:03.477003 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d47b1-f6c8-45a0-b0e4-609d80684107 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:03.477922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d47b1-f6c8-45a0-b0e4-609d80684107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:03.481058 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 934/1815] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:39:03.491595 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d47b1-f6c8-45a0-b0e4-609d80684107 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:03.494484 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c572203-a2de-4822-8cd0-a025fe8857a7 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:03.517280 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c572203-a2de-4822-8cd0-a025fe8857a7 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:03.519431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 882/1816] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:03.528956 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d47b1-f6c8-45a0-b0e4-609d80684107 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:03.537912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b705ff7-6e72-41a2-82bf-40065ebd22d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:03.538858 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b705ff7-6e72-41a2-82bf-40065ebd22d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:03.539583 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b705ff7-6e72-41a2-82bf-40065ebd22d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:03.539863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b705ff7-6e72-41a2-82bf-40065ebd22d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:03.540650 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b705ff7-6e72-41a2-82bf-40065ebd22d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:03.545859 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b705ff7-6e72-41a2-82bf-40065ebd22d2 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:03.555625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d47b1-f6c8-45a0-b0e4-609d80684107 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:03.589704 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb77b1-3571-434f-a3b6-4a8d7a17a5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:03.589704 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb77b1-3571-434f-a3b6-4a8d7a17a5bd admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:03.589704 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb77b1-3571-434f-a3b6-4a8d7a17a5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:03.589704 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb77b1-3571-434f-a3b6-4a8d7a17a5bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:03.594496 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb77b1-3571-434f-a3b6-4a8d7a17a5bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:03.607406 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb77b1-3571-434f-a3b6-4a8d7a17a5bd admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:03.662517 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 883/1817] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:03.666108 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb77b1-3571-434f-a3b6-4a8d7a17a5bd admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:03.667229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 935/1818] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:03 2022] POST /identity/v3/projects => generated 369 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:03.675106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b705ff7-6e72-41a2-82bf-40065ebd22d2 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:03.694501 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb77b1-3571-434f-a3b6-4a8d7a17a5bd admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:03.799750 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b705ff7-6e72-41a2-82bf-40065ebd22d2 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:03.810104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5022458a-d920-4322-a0b6-338063ae556d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:03.811202 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022458a-d920-4322-a0b6-338063ae556d admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:03.812156 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022458a-d920-4322-a0b6-338063ae556d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:03.812498 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022458a-d920-4322-a0b6-338063ae556d admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:03.814124 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5022458a-d920-4322-a0b6-338063ae556d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:03.817385 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5022458a-d920-4322-a0b6-338063ae556d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:03.852034 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c696729-834b-45c5-86c1-892fdf6420e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:03.852034 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c696729-834b-45c5-86c1-892fdf6420e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:03.852034 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c696729-834b-45c5-86c1-892fdf6420e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:03.852034 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c696729-834b-45c5-86c1-892fdf6420e5 admin admin] PATH_INFO: `/v3/projects/0c0895235fbc47298ff2372ce93fd24e/users/86b04f12693c492198c01274bbc83571/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:03.867125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c696729-834b-45c5-86c1-892fdf6420e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c0895235fbc47298ff2372ce93fd24e, user_id=86b04f12693c492198c01274bbc83571, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:03.869807 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c696729-834b-45c5-86c1-892fdf6420e5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:03.906216 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 884/1819] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:03 2022] PUT /identity/v3/projects/0c0895235fbc47298ff2372ce93fd24e/users/86b04f12693c492198c01274bbc83571/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:03.906216 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c696729-834b-45c5-86c1-892fdf6420e5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:03.909129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 936/1820] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:03 2022] POST /identity/v3/users => generated 285 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:03.912073 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a3607-a125-499c-baf2-8686b1adb421 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:03.912596 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a3607-a125-499c-baf2-8686b1adb421 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:03.913798 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a3607-a125-499c-baf2-8686b1adb421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:03.991509 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-dd9a3607-a125-499c-baf2-8686b1adb421 None None] MFA Rules not processed for user `86b04f12693c492198c01274bbc83571`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:03.993909 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-dd9a3607-a125-499c-baf2-8686b1adb421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:04.045062 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:04.046487 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c696729-834b-45c5-86c1-892fdf6420e5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:04.055069 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5a558-2b0c-439c-8361-5484ac5a38a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:04.055865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5a558-2b0c-439c-8361-5484ac5a38a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:04.057256 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5a558-2b0c-439c-8361-5484ac5a38a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:04.057256 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5a558-2b0c-439c-8361-5484ac5a38a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:04.057256 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae5a558-2b0c-439c-8361-5484ac5a38a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:04.059243 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae5a558-2b0c-439c-8361-5484ac5a38a8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:04.090537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 885/1821] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:04.091548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5a558-2b0c-439c-8361-5484ac5a38a8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:04.094671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7ae5a558-2b0c-439c-8361-5484ac5a38a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:04.169695 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 937/1822] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:03 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:04.170944 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a3607-a125-499c-baf2-8686b1adb421 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:04.191932 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a3607-a125-499c-baf2-8686b1adb421 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:04.212432 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9abd0-8294-426d-b1c1-cd0860e9608b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:04.213267 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9abd0-8294-426d-b1c1-cd0860e9608b admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:04.213873 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9abd0-8294-426d-b1c1-cd0860e9608b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:04.214213 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9abd0-8294-426d-b1c1-cd0860e9608b admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:04.215332 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9abd0-8294-426d-b1c1-cd0860e9608b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:04.219963 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9abd0-8294-426d-b1c1-cd0860e9608b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:04.229777 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:04.243087 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 938/1823] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:04.244456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9abd0-8294-426d-b1c1-cd0860e9608b admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:04.256820 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9abd0-8294-426d-b1c1-cd0860e9608b admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:04.262379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb097ac9-3038-4307-a378-f7b6e4f83b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:04.263444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb097ac9-3038-4307-a378-f7b6e4f83b74 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:04.263736 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb097ac9-3038-4307-a378-f7b6e4f83b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:04.264044 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb097ac9-3038-4307-a378-f7b6e4f83b74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:04.264869 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb097ac9-3038-4307-a378-f7b6e4f83b74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:04.267161 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb097ac9-3038-4307-a378-f7b6e4f83b74 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:04.269200 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5a558-2b0c-439c-8361-5484ac5a38a8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:04.275860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17032eee-8bd0-4157-9b8c-9a33d93d8a63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:04.276476 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17032eee-8bd0-4157-9b8c-9a33d93d8a63 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:04.276764 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17032eee-8bd0-4157-9b8c-9a33d93d8a63 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:04.277101 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17032eee-8bd0-4157-9b8c-9a33d93d8a63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:04.288014 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17032eee-8bd0-4157-9b8c-9a33d93d8a63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:04.290294 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17032eee-8bd0-4157-9b8c-9a33d93d8a63 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:04.302928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 939/1824] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:04 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:04.302928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb097ac9-3038-4307-a378-f7b6e4f83b74 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:04.313278 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 886/1825] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:04.314270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17032eee-8bd0-4157-9b8c-9a33d93d8a63 service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:04.331849 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17032eee-8bd0-4157-9b8c-9a33d93d8a63 service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:04.394278 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb097ac9-3038-4307-a378-f7b6e4f83b74 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:04.402424 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb14cc-73d8-4398-b01f-838a20ed7e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:04.404010 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb14cc-73d8-4398-b01f-838a20ed7e38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:04.404463 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb14cc-73d8-4398-b01f-838a20ed7e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:04.404762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb14cc-73d8-4398-b01f-838a20ed7e38 admin admin] PATH_INFO: `/v3/projects/4a54e967052b4265b99881a174443298/users/8ec9dde5ec9c4a54abae498115c1f9ff/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:04.405068 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8140460-fbf9-4128-ab11-82822155471f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:04.405821 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8140460-fbf9-4128-ab11-82822155471f admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:04.406115 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8140460-fbf9-4128-ab11-82822155471f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:04.406396 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8140460-fbf9-4128-ab11-82822155471f admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:04.407201 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8140460-fbf9-4128-ab11-82822155471f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:04.410549 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8140460-fbf9-4128-ab11-82822155471f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:04.427046 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb14cc-73d8-4398-b01f-838a20ed7e38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a54e967052b4265b99881a174443298, user_id=8ec9dde5ec9c4a54abae498115c1f9ff, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:04.430096 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb14cc-73d8-4398-b01f-838a20ed7e38 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:04.435124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 887/1826] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:04 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:04.439224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8140460-fbf9-4128-ab11-82822155471f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:04.452248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 940/1827] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:04 2022] PUT /identity/v3/projects/4a54e967052b4265b99881a174443298/users/8ec9dde5ec9c4a54abae498115c1f9ff/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:04.456410 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8140460-fbf9-4128-ab11-82822155471f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:04.460141 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530f4a3-95d7-4a8f-bb56-3498eaefe5ed None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:04.463129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530f4a3-95d7-4a8f-bb56-3498eaefe5ed None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:04.463691 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530f4a3-95d7-4a8f-bb56-3498eaefe5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:04.505111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-c530f4a3-95d7-4a8f-bb56-3498eaefe5ed None None] MFA Rules not processed for user `8ec9dde5ec9c4a54abae498115c1f9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:04.506535 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c530f4a3-95d7-4a8f-bb56-3498eaefe5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:04.544948 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2dcb70-451d-480b-a2e4-370119c34656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:04.545655 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2dcb70-451d-480b-a2e4-370119c34656 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:04.545944 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2dcb70-451d-480b-a2e4-370119c34656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:04.546225 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2dcb70-451d-480b-a2e4-370119c34656 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:04.550751 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2dcb70-451d-480b-a2e4-370119c34656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:04.554171 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2dcb70-451d-480b-a2e4-370119c34656 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:04.567524 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 888/1828] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:04.572586 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2dcb70-451d-480b-a2e4-370119c34656 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:04.584689 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2dcb70-451d-480b-a2e4-370119c34656 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:04.591995 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627c401-c2e2-4ebc-82d6-e11f572b3857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:04.592923 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627c401-c2e2-4ebc-82d6-e11f572b3857 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:04.593211 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627c401-c2e2-4ebc-82d6-e11f572b3857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:04.593590 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627c401-c2e2-4ebc-82d6-e11f572b3857 admin admin] PATH_INFO: `/v3/projects/f653d405e6ef4e048536717aad8591a1/users/56834de3f57945ce817eb223d76511d0/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:04.610588 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 941/1829] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:04.615065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c530f4a3-95d7-4a8f-bb56-3498eaefe5ed None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:04.615194 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627c401-c2e2-4ebc-82d6-e11f572b3857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f653d405e6ef4e048536717aad8591a1, user_id=56834de3f57945ce817eb223d76511d0, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:04.618487 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627c401-c2e2-4ebc-82d6-e11f572b3857 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:04.627753 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c530f4a3-95d7-4a8f-bb56-3498eaefe5ed None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:04.634167 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b03d4-5eae-48f3-99e2-69f10db07510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:04.635080 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b03d4-5eae-48f3-99e2-69f10db07510 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:04.635416 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b03d4-5eae-48f3-99e2-69f10db07510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:04.635761 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b03d4-5eae-48f3-99e2-69f10db07510 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:04.636738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b03d4-5eae-48f3-99e2-69f10db07510 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:04.638955 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b03d4-5eae-48f3-99e2-69f10db07510 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:04.643060 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:04.652971 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 889/1830] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:04 2022] PUT /identity/v3/projects/f653d405e6ef4e048536717aad8591a1/users/56834de3f57945ce817eb223d76511d0/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:04.671126 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627c401-c2e2-4ebc-82d6-e11f572b3857 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:04.683546 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 942/1831] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:04 2022] POST /identity/v3/projects => generated 367 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:04.692668 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b03d4-5eae-48f3-99e2-69f10db07510 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:04.768975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:04.821745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627c401-c2e2-4ebc-82d6-e11f572b3857 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:04.822911 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b03d4-5eae-48f3-99e2-69f10db07510 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:04.829377 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870911d-4340-4dc8-b877-b8daca7130f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:04.830164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870911d-4340-4dc8-b877-b8daca7130f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:04.830459 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870911d-4340-4dc8-b877-b8daca7130f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:04.830791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870911d-4340-4dc8-b877-b8daca7130f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:04.831610 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870911d-4340-4dc8-b877-b8daca7130f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:04.833639 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870911d-4340-4dc8-b877-b8daca7130f6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:04.837890 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890569a2-764a-410b-96fc-caac22f1d06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:04.838889 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890569a2-764a-410b-96fc-caac22f1d06e admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:04.839122 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890569a2-764a-410b-96fc-caac22f1d06e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:04.839468 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890569a2-764a-410b-96fc-caac22f1d06e admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:04.840335 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890569a2-764a-410b-96fc-caac22f1d06e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:04.843263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890569a2-764a-410b-96fc-caac22f1d06e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:04.846029 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 890/1832] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:04.847141 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870911d-4340-4dc8-b877-b8daca7130f6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:04.851081 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d870911d-4340-4dc8-b877-b8daca7130f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:04.870387 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870911d-4340-4dc8-b877-b8daca7130f6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:04.876046 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3be684-1b39-4af8-b2c1-0e1bb62d67d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:04.877088 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3be684-1b39-4af8-b2c1-0e1bb62d67d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:04.877388 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3be684-1b39-4af8-b2c1-0e1bb62d67d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:04.877675 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3be684-1b39-4af8-b2c1-0e1bb62d67d3 admin admin] PATH_INFO: `/v3/projects/2c447b607152422aa828c2689eb0b824` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:04.878987 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3be684-1b39-4af8-b2c1-0e1bb62d67d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c447b607152422aa828c2689eb0b824)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:04.881281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3be684-1b39-4af8-b2c1-0e1bb62d67d3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:04.888025 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 943/1833] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:04 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:04.888025 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890569a2-764a-410b-96fc-caac22f1d06e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:04.897674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890569a2-764a-410b-96fc-caac22f1d06e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:04.903133 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4e9f8-0ecf-44e5-b1fb-c8e887054953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:04.904136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4e9f8-0ecf-44e5-b1fb-c8e887054953 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:04.904418 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4e9f8-0ecf-44e5-b1fb-c8e887054953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:04.904816 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4e9f8-0ecf-44e5-b1fb-c8e887054953 admin admin] PATH_INFO: `/v3/projects/f653d405e6ef4e048536717aad8591a1/users/56834de3f57945ce817eb223d76511d0/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:04.907250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e4e9f8-0ecf-44e5-b1fb-c8e887054953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f653d405e6ef4e048536717aad8591a1, user_id=56834de3f57945ce817eb223d76511d0, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:04.910434 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e4e9f8-0ecf-44e5-b1fb-c8e887054953 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:04.923097 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 944/1834] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:04 2022] PUT /identity/v3/projects/f653d405e6ef4e048536717aad8591a1/users/56834de3f57945ce817eb223d76511d0/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:04.932032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4e9f8-0ecf-44e5-b1fb-c8e887054953 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:04.940630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dc3be684-1b39-4af8-b2c1-0e1bb62d67d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c447b607152422aa828c2689eb0b824', 'request_id': 'req-dc3be684-1b39-4af8-b2c1-0e1bb62d67d3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:04.952396 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dc3be684-1b39-4af8-b2c1-0e1bb62d67d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c447b607152422aa828c2689eb0b824', 'request_id': 'req-dc3be684-1b39-4af8-b2c1-0e1bb62d67d3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:04.959675 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dc3be684-1b39-4af8-b2c1-0e1bb62d67d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c447b607152422aa828c2689eb0b824', 'request_id': 'req-dc3be684-1b39-4af8-b2c1-0e1bb62d67d3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:04.975666 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dc3be684-1b39-4af8-b2c1-0e1bb62d67d3 admin admin] The token cache is being invalidate because project 2c447b607152422aa828c2689eb0b824 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:04.975851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dc3be684-1b39-4af8-b2c1-0e1bb62d67d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:04.977169 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dc3be684-1b39-4af8-b2c1-0e1bb62d67d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:04.983272 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 891/1835] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:04 2022] DELETE /identity/v3/projects/2c447b607152422aa828c2689eb0b824 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:04.984802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdbe70-8a82-42b7-b67e-99aa89f71dfb None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:04.985087 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdbe70-8a82-42b7-b67e-99aa89f71dfb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:04.985596 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdbe70-8a82-42b7-b67e-99aa89f71dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:05.024172 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-71cdbe70-8a82-42b7-b67e-99aa89f71dfb None None] MFA Rules not processed for user `56834de3f57945ce817eb223d76511d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:05.025475 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-71cdbe70-8a82-42b7-b67e-99aa89f71dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:05.035319 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4e9f8-0ecf-44e5-b1fb-c8e887054953 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:05.130357 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8223a-6ab3-49da-9d57-5ca5764c3457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:05.131669 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8223a-6ab3-49da-9d57-5ca5764c3457 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:05.131964 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8223a-6ab3-49da-9d57-5ca5764c3457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:05.133429 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8223a-6ab3-49da-9d57-5ca5764c3457 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:05.136549 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 892/1836] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:04 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:05.137500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8223a-6ab3-49da-9d57-5ca5764c3457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:05.139973 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8223a-6ab3-49da-9d57-5ca5764c3457 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:05.141022 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d908e4-ffcd-4d9c-9216-77c5e0d04ac1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:05.141334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d908e4-ffcd-4d9c-9216-77c5e0d04ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:05.141618 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d908e4-ffcd-4d9c-9216-77c5e0d04ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:05.160457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 945/1837] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:05.160457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8223a-6ab3-49da-9d57-5ca5764c3457 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:05.160703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-00d8223a-6ab3-49da-9d57-5ca5764c3457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:05.169209 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:05.182659 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-04d908e4-ffcd-4d9c-9216-77c5e0d04ac1 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:05.187164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-04d908e4-ffcd-4d9c-9216-77c5e0d04ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:05.208035 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 893/1838] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:05.210203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d908e4-ffcd-4d9c-9216-77c5e0d04ac1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:05.212835 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-04d908e4-ffcd-4d9c-9216-77c5e0d04ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:05.233999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d908e4-ffcd-4d9c-9216-77c5e0d04ac1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:05.243618 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c482c-825a-4c46-99c9-2a7e7611193f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:05.244766 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c482c-825a-4c46-99c9-2a7e7611193f admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:05.245148 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c482c-825a-4c46-99c9-2a7e7611193f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:05.245502 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c482c-825a-4c46-99c9-2a7e7611193f admin admin] PATH_INFO: `/v3/projects/93ff53c618904fc4980d0e9459124048/users/00d9db2335b74ec3a22584b426cfcc3a/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:05.258840 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8223a-6ab3-49da-9d57-5ca5764c3457 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:05.262807 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c00156-be54-4347-ac69-0d1199283d0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:05.263541 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c00156-be54-4347-ac69-0d1199283d0d service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:05.263894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c00156-be54-4347-ac69-0d1199283d0d service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:05.264306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c00156-be54-4347-ac69-0d1199283d0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:05.271227 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c482c-825a-4c46-99c9-2a7e7611193f admin admin] RBAC: Authorizing `identity:create_grant(project_id=93ff53c618904fc4980d0e9459124048, user_id=00d9db2335b74ec3a22584b426cfcc3a, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:05.274131 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c482c-825a-4c46-99c9-2a7e7611193f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:05.274464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c00156-be54-4347-ac69-0d1199283d0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:05.276114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c00156-be54-4347-ac69-0d1199283d0d service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:05.287641 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 946/1839] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:05 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:05.289848 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54a1a3-1e89-4c64-9fc7-6667068becc3 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:05.290146 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54a1a3-1e89-4c64-9fc7-6667068becc3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:05.290425 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54a1a3-1e89-4c64-9fc7-6667068becc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:05.299052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 894/1840] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:05 2022] PUT /identity/v3/projects/93ff53c618904fc4980d0e9459124048/users/00d9db2335b74ec3a22584b426cfcc3a/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:05.302878 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c482c-825a-4c46-99c9-2a7e7611193f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:05.321450 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5a54a1a3-1e89-4c64-9fc7-6667068becc3 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:05.322861 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5a54a1a3-1e89-4c64-9fc7-6667068becc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:05.341508 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:05.423684 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c482c-825a-4c46-99c9-2a7e7611193f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:05.425193 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 947/1841] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:05.428053 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54a1a3-1e89-4c64-9fc7-6667068becc3 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:05.429124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32325a28-cd17-457c-9b57-53a2ea402494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:05.429858 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32325a28-cd17-457c-9b57-53a2ea402494 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:05.430174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32325a28-cd17-457c-9b57-53a2ea402494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:05.430452 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32325a28-cd17-457c-9b57-53a2ea402494 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:05.431244 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32325a28-cd17-457c-9b57-53a2ea402494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:05.433044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32325a28-cd17-457c-9b57-53a2ea402494 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:05.448064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 895/1842] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:05.449852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32325a28-cd17-457c-9b57-53a2ea402494 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:05.465352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32325a28-cd17-457c-9b57-53a2ea402494 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:05.472158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad2d5c-5be4-4d5d-91cd-bfea727a420b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:05.473215 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad2d5c-5be4-4d5d-91cd-bfea727a420b admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:05.473877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad2d5c-5be4-4d5d-91cd-bfea727a420b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:05.474688 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad2d5c-5be4-4d5d-91cd-bfea727a420b admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:05.476050 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad2d5c-5be4-4d5d-91cd-bfea727a420b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:39:05.476898 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad2d5c-5be4-4d5d-91cd-bfea727a420b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:05.478844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad2d5c-5be4-4d5d-91cd-bfea727a420b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:05.500158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 896/1843] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:39:05.500158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad2d5c-5be4-4d5d-91cd-bfea727a420b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:05.503302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54a1a3-1e89-4c64-9fc7-6667068becc3 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:05.507508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f4e2d-02b5-401e-a6df-f5524e46da68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:05.507735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad2d5c-5be4-4d5d-91cd-bfea727a420b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:05.508279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f4e2d-02b5-401e-a6df-f5524e46da68 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:05.508582 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f4e2d-02b5-401e-a6df-f5524e46da68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:05.508838 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f4e2d-02b5-401e-a6df-f5524e46da68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:05.510625 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5b4f4e2d-02b5-401e-a6df-f5524e46da68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:05.512599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee100a4d-bea9-4496-af0a-fef42428a327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:05.513479 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee100a4d-bea9-4496-af0a-fef42428a327 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:05.513765 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee100a4d-bea9-4496-af0a-fef42428a327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:05.514045 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee100a4d-bea9-4496-af0a-fef42428a327 admin admin] PATH_INFO: `/v3/projects/93ff53c618904fc4980d0e9459124048/users/00d9db2335b74ec3a22584b426cfcc3a/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:05.516020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee100a4d-bea9-4496-af0a-fef42428a327 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93ff53c618904fc4980d0e9459124048, user_id=00d9db2335b74ec3a22584b426cfcc3a, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:05.518626 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee100a4d-bea9-4496-af0a-fef42428a327 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:05.529420 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 897/1844] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:05 2022] PUT /identity/v3/projects/93ff53c618904fc4980d0e9459124048/users/00d9db2335b74ec3a22584b426cfcc3a/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:05.530191 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee100a4d-bea9-4496-af0a-fef42428a327 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:05.577667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4f4e2d-02b5-401e-a6df-f5524e46da68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:05.579196 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4f4e2d-02b5-401e-a6df-f5524e46da68 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:05.588993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 948/1845] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:05 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:05.590394 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ff63d-4cff-46af-ac20-35d6a67b1dd0 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:05.590839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee100a4d-bea9-4496-af0a-fef42428a327 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:05.591008 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ff63d-4cff-46af-ac20-35d6a67b1dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:05.591484 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ff63d-4cff-46af-ac20-35d6a67b1dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:05.595571 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea68869-5341-4d6a-ba01-bd4bc33e92b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:05.596231 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea68869-5341-4d6a-ba01-bd4bc33e92b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:05.596522 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea68869-5341-4d6a-ba01-bd4bc33e92b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:05.596804 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea68869-5341-4d6a-ba01-bd4bc33e92b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:05.597631 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea68869-5341-4d6a-ba01-bd4bc33e92b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:05.598143 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:05.600531 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea68869-5341-4d6a-ba01-bd4bc33e92b4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:05.627229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-e05ff63d-4cff-46af-ac20-35d6a67b1dd0 None None] MFA Rules not processed for user `00d9db2335b74ec3a22584b426cfcc3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:05.628451 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e05ff63d-4cff-46af-ac20-35d6a67b1dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:05.632273 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 898/1846] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:05 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:05.636572 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea68869-5341-4d6a-ba01-bd4bc33e92b4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:05.714684 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea68869-5341-4d6a-ba01-bd4bc33e92b4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:05.717395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 949/1847] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:05 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:05.720084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edc82f-55d2-4fa9-a6e5-14c0031adb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:05.720702 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edc82f-55d2-4fa9-a6e5-14c0031adb9b admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:05.721060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edc82f-55d2-4fa9-a6e5-14c0031adb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:05.721588 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edc82f-55d2-4fa9-a6e5-14c0031adb9b admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:05.722319 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60edc82f-55d2-4fa9-a6e5-14c0031adb9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:05.724453 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60edc82f-55d2-4fa9-a6e5-14c0031adb9b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:05.730543 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ff63d-4cff-46af-ac20-35d6a67b1dd0 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:05.746837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:05.752673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 899/1848] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:05 2022] POST /identity/v3/users => generated 283 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:05.756961 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edc82f-55d2-4fa9-a6e5-14c0031adb9b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:05.767413 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edc82f-55d2-4fa9-a6e5-14c0031adb9b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:05.772535 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e94c8-a1fc-4773-97e5-d5b9c3f4ba50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:05.773880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e94c8-a1fc-4773-97e5-d5b9c3f4ba50 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:05.774242 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e94c8-a1fc-4773-97e5-d5b9c3f4ba50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:05.774593 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e94c8-a1fc-4773-97e5-d5b9c3f4ba50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:05.775456 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e94c8-a1fc-4773-97e5-d5b9c3f4ba50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:05.777190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e94c8-a1fc-4773-97e5-d5b9c3f4ba50 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:05.791684 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 900/1849] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:39:05.792552 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e94c8-a1fc-4773-97e5-d5b9c3f4ba50 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:05.794852 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ff63d-4cff-46af-ac20-35d6a67b1dd0 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:05.802617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e94c8-a1fc-4773-97e5-d5b9c3f4ba50 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:05.807745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cf568-e242-4da4-81aa-02f02472ee89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:05.808401 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cf568-e242-4da4-81aa-02f02472ee89 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:05.808695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cf568-e242-4da4-81aa-02f02472ee89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:05.808975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cf568-e242-4da4-81aa-02f02472ee89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:05.809704 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cf568-e242-4da4-81aa-02f02472ee89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:05.811338 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cf568-e242-4da4-81aa-02f02472ee89 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:05.822840 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 901/1850] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:05.827042 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cf568-e242-4da4-81aa-02f02472ee89 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:05.838579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cf568-e242-4da4-81aa-02f02472ee89 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:05.843312 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa8bc7-7d34-4128-9217-c87ef61403d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:05.844240 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa8bc7-7d34-4128-9217-c87ef61403d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:05.844513 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa8bc7-7d34-4128-9217-c87ef61403d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:05.844786 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa8bc7-7d34-4128-9217-c87ef61403d8 admin admin] PATH_INFO: `/v3/projects/fdd0e28e9b7a4e019ae3804d5bc3424f/users/4940be82d6074beb9c0b40895e87bcad/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:05.846005 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1b9ce-f90c-4788-8525-d4697312f1ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:05.846940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1b9ce-f90c-4788-8525-d4697312f1ff service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:05.847389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1b9ce-f90c-4788-8525-d4697312f1ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:05.847906 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1b9ce-f90c-4788-8525-d4697312f1ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:05.861425 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aa8bc7-7d34-4128-9217-c87ef61403d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdd0e28e9b7a4e019ae3804d5bc3424f, user_id=4940be82d6074beb9c0b40895e87bcad, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:05.869571 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aa8bc7-7d34-4128-9217-c87ef61403d8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:05.885316 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 902/1851] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:05 2022] PUT /identity/v3/projects/fdd0e28e9b7a4e019ae3804d5bc3424f/users/4940be82d6074beb9c0b40895e87bcad/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:05.895887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fe416-ab70-41c0-a8ab-5ceaac1c1ea9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:05.896177 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fe416-ab70-41c0-a8ab-5ceaac1c1ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:05.896457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fe416-ab70-41c0-a8ab-5ceaac1c1ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:05.919554 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:05.927965 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1b9ce-f90c-4788-8525-d4697312f1ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:05.929349 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-684fe416-ab70-41c0-a8ab-5ceaac1c1ea9 None None] MFA Rules not processed for user `4940be82d6074beb9c0b40895e87bcad`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:05.929631 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1b9ce-f90c-4788-8525-d4697312f1ff service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:05.930845 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-684fe416-ab70-41c0-a8ab-5ceaac1c1ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:05.952641 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 950/1852] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:05 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:39:06.017461 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 903/1853] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:05 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:39:06.021895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1b9ce-f90c-4788-8525-d4697312f1ff service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:06.033147 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1b9ce-f90c-4788-8525-d4697312f1ff service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:06.094971 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04bd1bd-a332-4d5d-80d9-6c845cd02e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:06.095646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bd1bd-a332-4d5d-80d9-6c845cd02e46 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:06.095927 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bd1bd-a332-4d5d-80d9-6c845cd02e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:06.096202 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bd1bd-a332-4d5d-80d9-6c845cd02e46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:06.096898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04bd1bd-a332-4d5d-80d9-6c845cd02e46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:06.098828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04bd1bd-a332-4d5d-80d9-6c845cd02e46 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:06.129892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 951/1854] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:06 2022] POST /identity/v3/projects => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:06.134560 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684fe416-ab70-41c0-a8ab-5ceaac1c1ea9 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:06.145284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684fe416-ab70-41c0-a8ab-5ceaac1c1ea9 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:06.239591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689d01c-cde7-4437-ae6d-d65ee4770209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:06.240617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689d01c-cde7-4437-ae6d-d65ee4770209 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:06.241062 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689d01c-cde7-4437-ae6d-d65ee4770209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:06.241217 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689d01c-cde7-4437-ae6d-d65ee4770209 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:06.242109 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689d01c-cde7-4437-ae6d-d65ee4770209 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:06.244507 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689d01c-cde7-4437-ae6d-d65ee4770209 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:06.303511 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 904/1855] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:06 2022] POST /identity/v3/users => generated 290 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:39:06.310930 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04bd1bd-a332-4d5d-80d9-6c845cd02e46 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:06.333803 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04bd1bd-a332-4d5d-80d9-6c845cd02e46 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:06.350774 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719e97e-c7e1-48d5-adad-e195621a4ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:06.352335 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719e97e-c7e1-48d5-adad-e195621a4ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:06.352624 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719e97e-c7e1-48d5-adad-e195621a4ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:06.352915 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719e97e-c7e1-48d5-adad-e195621a4ab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:06.355807 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719e97e-c7e1-48d5-adad-e195621a4ab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:06.358441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719e97e-c7e1-48d5-adad-e195621a4ab4 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:06.391140 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 952/1856] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:06.395357 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689d01c-cde7-4437-ae6d-d65ee4770209 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:06.410962 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689d01c-cde7-4437-ae6d-d65ee4770209 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:06.427199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd12261-dd88-405a-85b7-5cd0c10f21c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:06.428633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd12261-dd88-405a-85b7-5cd0c10f21c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:06.429103 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd12261-dd88-405a-85b7-5cd0c10f21c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:06.429599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd12261-dd88-405a-85b7-5cd0c10f21c7 admin admin] PATH_INFO: `/v3/projects/844ac7ef7d154666895857d591d2c08f/users/966e2e7af9e449dea77e6fffb77351d6/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:06.448634 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd12261-dd88-405a-85b7-5cd0c10f21c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=844ac7ef7d154666895857d591d2c08f, user_id=966e2e7af9e449dea77e6fffb77351d6, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:06.452242 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd12261-dd88-405a-85b7-5cd0c10f21c7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:06.485927 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 905/1857] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:06 2022] PUT /identity/v3/projects/844ac7ef7d154666895857d591d2c08f/users/966e2e7af9e449dea77e6fffb77351d6/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:06.489983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719e97e-c7e1-48d5-adad-e195621a4ab4 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:06.505320 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719e97e-c7e1-48d5-adad-e195621a4ab4 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:06.506835 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd12261-dd88-405a-85b7-5cd0c10f21c7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:06.509576 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fcd12261-dd88-405a-85b7-5cd0c10f21c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:06.524134 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:06.605694 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd12261-dd88-405a-85b7-5cd0c10f21c7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:06.616007 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1d364-3bc3-49a9-9920-fba9cd21302d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:06.620507 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1d364-3bc3-49a9-9920-fba9cd21302d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:06.621247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1d364-3bc3-49a9-9920-fba9cd21302d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:06.622785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1d364-3bc3-49a9-9920-fba9cd21302d admin admin] PATH_INFO: `/v3/users/0cd817ab2cab4ebeac149395f8fec5ec` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:06.627046 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1d364-3bc3-49a9-9920-fba9cd21302d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cd817ab2cab4ebeac149395f8fec5ec)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:06.643109 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1d364-3bc3-49a9-9920-fba9cd21302d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:06.648865 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66d51b-173a-4dea-9d3b-519216136c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:06.649780 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66d51b-173a-4dea-9d3b-519216136c4a admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:06.650240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66d51b-173a-4dea-9d3b-519216136c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:06.650778 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66d51b-173a-4dea-9d3b-519216136c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:06.651458 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e66d51b-173a-4dea-9d3b-519216136c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:06.653081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e66d51b-173a-4dea-9d3b-519216136c4a admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:06.668883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 953/1858] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:06.679457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66d51b-173a-4dea-9d3b-519216136c4a admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:06.709254 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66d51b-173a-4dea-9d3b-519216136c4a admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:06.712394 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad91542-2075-42c4-a136-fb3ec45a35f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:06.713908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad91542-2075-42c4-a136-fb3ec45a35f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:06.714052 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad91542-2075-42c4-a136-fb3ec45a35f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:06.714339 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad91542-2075-42c4-a136-fb3ec45a35f4 admin admin] PATH_INFO: `/v3/projects/844ac7ef7d154666895857d591d2c08f/users/966e2e7af9e449dea77e6fffb77351d6/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:06.716606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad91542-2075-42c4-a136-fb3ec45a35f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=844ac7ef7d154666895857d591d2c08f, user_id=966e2e7af9e449dea77e6fffb77351d6, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:06.719706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad91542-2075-42c4-a136-fb3ec45a35f4 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:06.741808 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 954/1859] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:06 2022] PUT /identity/v3/projects/844ac7ef7d154666895857d591d2c08f/users/966e2e7af9e449dea77e6fffb77351d6/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:06.753409 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6bd1d364-3bc3-49a9-9920-fba9cd21302d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cd817ab2cab4ebeac149395f8fec5ec'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:06.765152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b73e0-79b3-4886-b9c1-8fc4860eb135 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:06.765488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b73e0-79b3-4886-b9c1-8fc4860eb135 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:06.765760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b73e0-79b3-4886-b9c1-8fc4860eb135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:06.771307 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6bd1d364-3bc3-49a9-9920-fba9cd21302d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cd817ab2cab4ebeac149395f8fec5ec'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:06.785997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6bd1d364-3bc3-49a9-9920-fba9cd21302d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cd817ab2cab4ebeac149395f8fec5ec'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:06.808579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6bd1d364-3bc3-49a9-9920-fba9cd21302d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cd817ab2cab4ebeac149395f8fec5ec'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:06.810080 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6bd1d364-3bc3-49a9-9920-fba9cd21302d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cd817ab2cab4ebeac149395f8fec5ec'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:06.823457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 906/1860] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:06 2022] DELETE /identity/v3/users/0cd817ab2cab4ebeac149395f8fec5ec => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:06.826316 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4c5c3-11c2-4b20-8d63-a1e54a7887ad None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:06.826610 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4c5c3-11c2-4b20-8d63-a1e54a7887ad None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:06.826982 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4c5c3-11c2-4b20-8d63-a1e54a7887ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:06.840614 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-674b73e0-79b3-4886-b9c1-8fc4860eb135 None None] MFA Rules not processed for user `966e2e7af9e449dea77e6fffb77351d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:06.841852 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-674b73e0-79b3-4886-b9c1-8fc4860eb135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:06.859823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c2a4c5c3-11c2-4b20-8d63-a1e54a7887ad None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:06.861703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c2a4c5c3-11c2-4b20-8d63-a1e54a7887ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:06.908909 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:06.967177 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 955/1861] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:06 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:06.993956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b73e0-79b3-4886-b9c1-8fc4860eb135 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:06.995466 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-674b73e0-79b3-4886-b9c1-8fc4860eb135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:07.023433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 907/1862] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:07.033263 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4c5c3-11c2-4b20-8d63-a1e54a7887ad None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:07.034722 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c2a4c5c3-11c2-4b20-8d63-a1e54a7887ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:07.069404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:07.082691 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b73e0-79b3-4886-b9c1-8fc4860eb135 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:07.086781 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ceb0e-3bba-4417-b97e-54663596c1c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:07.087440 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ceb0e-3bba-4417-b97e-54663596c1c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:07.087977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ceb0e-3bba-4417-b97e-54663596c1c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:07.088166 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ceb0e-3bba-4417-b97e-54663596c1c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:07.098748 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ceb0e-3bba-4417-b97e-54663596c1c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:07.100233 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ceb0e-3bba-4417-b97e-54663596c1c2 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:07.105703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4c5c3-11c2-4b20-8d63-a1e54a7887ad None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:07.112187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826305a-1b45-43bb-aacd-126a104573ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:07.112296 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 956/1863] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:06 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:39:07.112479 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826305a-1b45-43bb-aacd-126a104573ad service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:07.112479 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826305a-1b45-43bb-aacd-126a104573ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:07.112479 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826305a-1b45-43bb-aacd-126a104573ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:07.122584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6826305a-1b45-43bb-aacd-126a104573ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:07.124120 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6826305a-1b45-43bb-aacd-126a104573ad service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:07.136850 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 908/1864] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:39:07.297250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ceb0e-3bba-4417-b97e-54663596c1c2 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:07.307921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ceb0e-3bba-4417-b97e-54663596c1c2 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:07.311844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71810567-9597-4cf5-bff6-0d374c154f9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:07.312488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71810567-9597-4cf5-bff6-0d374c154f9a service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:07.312769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71810567-9597-4cf5-bff6-0d374c154f9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:07.313141 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71810567-9597-4cf5-bff6-0d374c154f9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:07.315322 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-71810567-9597-4cf5-bff6-0d374c154f9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:07.380562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71810567-9597-4cf5-bff6-0d374c154f9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:07.382014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71810567-9597-4cf5-bff6-0d374c154f9a service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:07.392079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 957/1865] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:07 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:07.454064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826305a-1b45-43bb-aacd-126a104573ad service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:07.457197 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6826305a-1b45-43bb-aacd-126a104573ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:07.474926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826305a-1b45-43bb-aacd-126a104573ad service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:07.483600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b6d320-784e-4617-9eaa-b0a77d83c43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:07.484247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b6d320-784e-4617-9eaa-b0a77d83c43a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:07.484590 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b6d320-784e-4617-9eaa-b0a77d83c43a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:07.484978 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b6d320-784e-4617-9eaa-b0a77d83c43a admin admin] PATH_INFO: `/v3/projects/12aeb512b98a4ac0acf1ea520b404122` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:07.487163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b6d320-784e-4617-9eaa-b0a77d83c43a admin admin] RBAC: Authorizing `identity:delete_project(project_id=12aeb512b98a4ac0acf1ea520b404122)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:07.489588 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b6d320-784e-4617-9eaa-b0a77d83c43a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:07.571159 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-66b6d320-784e-4617-9eaa-b0a77d83c43a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12aeb512b98a4ac0acf1ea520b404122', 'request_id': 'req-66b6d320-784e-4617-9eaa-b0a77d83c43a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:07.580097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-66b6d320-784e-4617-9eaa-b0a77d83c43a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12aeb512b98a4ac0acf1ea520b404122', 'request_id': 'req-66b6d320-784e-4617-9eaa-b0a77d83c43a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:07.590041 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-66b6d320-784e-4617-9eaa-b0a77d83c43a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12aeb512b98a4ac0acf1ea520b404122', 'request_id': 'req-66b6d320-784e-4617-9eaa-b0a77d83c43a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:07.607594 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-66b6d320-784e-4617-9eaa-b0a77d83c43a admin admin] The token cache is being invalidate because project 12aeb512b98a4ac0acf1ea520b404122 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:07.608007 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-66b6d320-784e-4617-9eaa-b0a77d83c43a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:07.608959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-66b6d320-784e-4617-9eaa-b0a77d83c43a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:07.611197 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 909/1866] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:07 2022] DELETE /identity/v3/projects/12aeb512b98a4ac0acf1ea520b404122 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:07.615938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71810567-9597-4cf5-bff6-0d374c154f9a service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:07.633711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71810567-9597-4cf5-bff6-0d374c154f9a service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:07.744917 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77123d4-372e-44e8-a219-1d277864ff3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:07.745967 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77123d4-372e-44e8-a219-1d277864ff3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:07.746201 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77123d4-372e-44e8-a219-1d277864ff3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:07.746548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77123d4-372e-44e8-a219-1d277864ff3a admin admin] PATH_INFO: `/v3/users/0d9fa91ba9084d0a830c54a1d8b56aa9` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:07.747988 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77123d4-372e-44e8-a219-1d277864ff3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d9fa91ba9084d0a830c54a1d8b56aa9)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:07.764645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77123d4-372e-44e8-a219-1d277864ff3a admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:07.816017 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:07.868592 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c77123d4-372e-44e8-a219-1d277864ff3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d9fa91ba9084d0a830c54a1d8b56aa9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:07.894468 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c77123d4-372e-44e8-a219-1d277864ff3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d9fa91ba9084d0a830c54a1d8b56aa9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:07.895198 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c77123d4-372e-44e8-a219-1d277864ff3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d9fa91ba9084d0a830c54a1d8b56aa9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:07.922192 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c77123d4-372e-44e8-a219-1d277864ff3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d9fa91ba9084d0a830c54a1d8b56aa9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:07.924066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c77123d4-372e-44e8-a219-1d277864ff3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d9fa91ba9084d0a830c54a1d8b56aa9'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:07.941357 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 958/1867] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:07 2022] DELETE /identity/v3/users/0d9fa91ba9084d0a830c54a1d8b56aa9 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:08.296998 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b6d320-784e-4617-9eaa-b0a77d83c43a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:08.299081 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-66b6d320-784e-4617-9eaa-b0a77d83c43a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:08.317546 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b6d320-784e-4617-9eaa-b0a77d83c43a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:08.378956 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb16ff-b286-4855-b86d-f53757d5c2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:08.379843 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb16ff-b286-4855-b86d-f53757d5c2b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:08.380134 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb16ff-b286-4855-b86d-f53757d5c2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:08.380413 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb16ff-b286-4855-b86d-f53757d5c2b6 admin admin] PATH_INFO: `/v3/projects/82cb9f1f05af4e50805cce3aff7964a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:08.382948 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb16ff-b286-4855-b86d-f53757d5c2b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82cb9f1f05af4e50805cce3aff7964a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:08.384963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb16ff-b286-4855-b86d-f53757d5c2b6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:08.437155 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8fbb16ff-b286-4855-b86d-f53757d5c2b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82cb9f1f05af4e50805cce3aff7964a1', 'request_id': 'req-8fbb16ff-b286-4855-b86d-f53757d5c2b6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:08.442292 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:08.446276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8fbb16ff-b286-4855-b86d-f53757d5c2b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82cb9f1f05af4e50805cce3aff7964a1', 'request_id': 'req-8fbb16ff-b286-4855-b86d-f53757d5c2b6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:08.455895 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8fbb16ff-b286-4855-b86d-f53757d5c2b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82cb9f1f05af4e50805cce3aff7964a1', 'request_id': 'req-8fbb16ff-b286-4855-b86d-f53757d5c2b6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:08.492218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8fbb16ff-b286-4855-b86d-f53757d5c2b6 admin admin] The token cache is being invalidate because project 82cb9f1f05af4e50805cce3aff7964a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:08.492218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8fbb16ff-b286-4855-b86d-f53757d5c2b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:08.492218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8fbb16ff-b286-4855-b86d-f53757d5c2b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:08.492218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 910/1868] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:08 2022] DELETE /identity/v3/projects/82cb9f1f05af4e50805cce3aff7964a1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:08.495142 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77123d4-372e-44e8-a219-1d277864ff3a admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:08.496777 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c77123d4-372e-44e8-a219-1d277864ff3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:08.510225 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77123d4-372e-44e8-a219-1d277864ff3a admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:08.584322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb16ff-b286-4855-b86d-f53757d5c2b6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:08.587351 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8fbb16ff-b286-4855-b86d-f53757d5c2b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:08.592574 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9fb11-0c0a-4634-860c-0b630bb2fab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:08.593493 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9fb11-0c0a-4634-860c-0b630bb2fab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:08.593832 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9fb11-0c0a-4634-860c-0b630bb2fab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:08.595908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9fb11-0c0a-4634-860c-0b630bb2fab8 admin admin] PATH_INFO: `/v3/users/27e384d4c6954153b87481b3bfd8956e` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:08.596669 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc9fb11-0c0a-4634-860c-0b630bb2fab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27e384d4c6954153b87481b3bfd8956e)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:08.602828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc9fb11-0c0a-4634-860c-0b630bb2fab8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:08.613150 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb16ff-b286-4855-b86d-f53757d5c2b6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:08.623701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a8b46-8430-4dc5-9fb1-77f38195a546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:08.626737 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a8b46-8430-4dc5-9fb1-77f38195a546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:08.626737 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a8b46-8430-4dc5-9fb1-77f38195a546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:08.626737 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a8b46-8430-4dc5-9fb1-77f38195a546 admin admin] PATH_INFO: `/v3/users/8ec9dde5ec9c4a54abae498115c1f9ff` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:08.631154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430a8b46-8430-4dc5-9fb1-77f38195a546 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ec9dde5ec9c4a54abae498115c1f9ff)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:08.633895 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430a8b46-8430-4dc5-9fb1-77f38195a546 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:08.667124 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-dcc9fb11-0c0a-4634-860c-0b630bb2fab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27e384d4c6954153b87481b3bfd8956e'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:08.677305 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-dcc9fb11-0c0a-4634-860c-0b630bb2fab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27e384d4c6954153b87481b3bfd8956e'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:08.678299 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-dcc9fb11-0c0a-4634-860c-0b630bb2fab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27e384d4c6954153b87481b3bfd8956e'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:08.692321 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-430a8b46-8430-4dc5-9fb1-77f38195a546 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ec9dde5ec9c4a54abae498115c1f9ff'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:08.693234 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-dcc9fb11-0c0a-4634-860c-0b630bb2fab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27e384d4c6954153b87481b3bfd8956e'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:08.693916 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-dcc9fb11-0c0a-4634-860c-0b630bb2fab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27e384d4c6954153b87481b3bfd8956e'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:08.701975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-430a8b46-8430-4dc5-9fb1-77f38195a546 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ec9dde5ec9c4a54abae498115c1f9ff'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:08.707964 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 959/1869] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:08 2022] DELETE /identity/v3/users/27e384d4c6954153b87481b3bfd8956e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:08.709467 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-430a8b46-8430-4dc5-9fb1-77f38195a546 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ec9dde5ec9c4a54abae498115c1f9ff'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:08.722663 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-430a8b46-8430-4dc5-9fb1-77f38195a546 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ec9dde5ec9c4a54abae498115c1f9ff'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:08.723477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-430a8b46-8430-4dc5-9fb1-77f38195a546 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ec9dde5ec9c4a54abae498115c1f9ff'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:08.729914 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 911/1870] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:08 2022] DELETE /identity/v3/users/8ec9dde5ec9c4a54abae498115c1f9ff => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:08.734355 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef7b6a-9d7b-4ebe-b60d-ba8a1e8adc05 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:08.734794 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef7b6a-9d7b-4ebe-b60d-ba8a1e8adc05 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:08.735143 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef7b6a-9d7b-4ebe-b60d-ba8a1e8adc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:08.753161 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-ecef7b6a-9d7b-4ebe-b60d-ba8a1e8adc05 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:08.754462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ecef7b6a-9d7b-4ebe-b60d-ba8a1e8adc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:08.766376 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:08.825698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 960/1871] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:08.836429 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a8b46-8430-4dc5-9fb1-77f38195a546 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:08.838520 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-430a8b46-8430-4dc5-9fb1-77f38195a546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:08.901641 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a8b46-8430-4dc5-9fb1-77f38195a546 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:08.905394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917eeaa-9ccd-4082-810f-83316ae15d40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:08.906075 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917eeaa-9ccd-4082-810f-83316ae15d40 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:08.906365 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917eeaa-9ccd-4082-810f-83316ae15d40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:08.906679 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917eeaa-9ccd-4082-810f-83316ae15d40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:08.915956 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917eeaa-9ccd-4082-810f-83316ae15d40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:08.918220 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917eeaa-9ccd-4082-810f-83316ae15d40 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:08.927812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 912/1872] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:09.073229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef7b6a-9d7b-4ebe-b60d-ba8a1e8adc05 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:09.075518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ecef7b6a-9d7b-4ebe-b60d-ba8a1e8adc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:09.097449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef7b6a-9d7b-4ebe-b60d-ba8a1e8adc05 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:09.105689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d2f56-de11-4d02-92f3-eb3543a66384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:09.106910 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d2f56-de11-4d02-92f3-eb3543a66384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:09.107421 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d2f56-de11-4d02-92f3-eb3543a66384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:09.108234 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d2f56-de11-4d02-92f3-eb3543a66384 admin admin] PATH_INFO: `/v3/projects/ffcebece2ea54b3ba4bf5de3660aa03d` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:09.110021 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d2f56-de11-4d02-92f3-eb3543a66384 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffcebece2ea54b3ba4bf5de3660aa03d)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:09.113754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d2f56-de11-4d02-92f3-eb3543a66384 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:09.189463 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b14d2f56-de11-4d02-92f3-eb3543a66384 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffcebece2ea54b3ba4bf5de3660aa03d', 'request_id': 'req-b14d2f56-de11-4d02-92f3-eb3543a66384'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:09.195892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b14d2f56-de11-4d02-92f3-eb3543a66384 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffcebece2ea54b3ba4bf5de3660aa03d', 'request_id': 'req-b14d2f56-de11-4d02-92f3-eb3543a66384'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:09.202632 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b14d2f56-de11-4d02-92f3-eb3543a66384 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffcebece2ea54b3ba4bf5de3660aa03d', 'request_id': 'req-b14d2f56-de11-4d02-92f3-eb3543a66384'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:09.218417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b14d2f56-de11-4d02-92f3-eb3543a66384 admin admin] The token cache is being invalidate because project ffcebece2ea54b3ba4bf5de3660aa03d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:09.218778 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b14d2f56-de11-4d02-92f3-eb3543a66384 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:09.219421 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b14d2f56-de11-4d02-92f3-eb3543a66384 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:09.226389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 961/1873] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:09 2022] DELETE /identity/v3/projects/ffcebece2ea54b3ba4bf5de3660aa03d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:09.228228 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce11a36-bc3f-44bb-a596-e1f69e6f30f3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:09.228651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce11a36-bc3f-44bb-a596-e1f69e6f30f3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:09.229005 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce11a36-bc3f-44bb-a596-e1f69e6f30f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:09.255713 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-bce11a36-bc3f-44bb-a596-e1f69e6f30f3 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:09.258609 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bce11a36-bc3f-44bb-a596-e1f69e6f30f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:09.325608 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:09.334794 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 913/1874] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:09.344709 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba642e-e8e9-48df-ae5c-3884ca85bd3f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:09.345193 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba642e-e8e9-48df-ae5c-3884ca85bd3f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:09.345509 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba642e-e8e9-48df-ae5c-3884ca85bd3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:09.375056 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-7dba642e-e8e9-48df-ae5c-3884ca85bd3f None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:09.376359 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7dba642e-e8e9-48df-ae5c-3884ca85bd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:09.399652 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce11a36-bc3f-44bb-a596-e1f69e6f30f3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:09.404594 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 962/1875] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:39:09.409505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce11a36-bc3f-44bb-a596-e1f69e6f30f3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:09.420652 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d0a81-d0a2-4a61-bb37-a809a92754ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:09.421351 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d0a81-d0a2-4a61-bb37-a809a92754ce admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:09.421644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d0a81-d0a2-4a61-bb37-a809a92754ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:09.421928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d0a81-d0a2-4a61-bb37-a809a92754ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:09.423216 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d0a81-d0a2-4a61-bb37-a809a92754ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:39:09.423542 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d0a81-d0a2-4a61-bb37-a809a92754ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:09.425492 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d0a81-d0a2-4a61-bb37-a809a92754ce admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:09.447070 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 914/1876] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:39:09.447631 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba642e-e8e9-48df-ae5c-3884ca85bd3f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:09.449524 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7dba642e-e8e9-48df-ae5c-3884ca85bd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:09.453633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d0a81-d0a2-4a61-bb37-a809a92754ce admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:09.465562 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d0a81-d0a2-4a61-bb37-a809a92754ce admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:09.467971 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba642e-e8e9-48df-ae5c-3884ca85bd3f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:09.472468 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c894ac-897f-4090-b601-662e5abb7e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:09.472745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3c525-07a2-4f25-a1dc-8bd2ef74de0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:09.473038 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c894ac-897f-4090-b601-662e5abb7e30 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:09.473328 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c894ac-897f-4090-b601-662e5abb7e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:09.473608 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c894ac-897f-4090-b601-662e5abb7e30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:09.474336 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c894ac-897f-4090-b601-662e5abb7e30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:09.474607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3c525-07a2-4f25-a1dc-8bd2ef74de0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:09.475261 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3c525-07a2-4f25-a1dc-8bd2ef74de0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:09.475983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3c525-07a2-4f25-a1dc-8bd2ef74de0c admin admin] PATH_INFO: `/v3/projects/4a54e967052b4265b99881a174443298` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:09.476311 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c894ac-897f-4090-b601-662e5abb7e30 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:09.478167 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3c525-07a2-4f25-a1dc-8bd2ef74de0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a54e967052b4265b99881a174443298)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:09.480115 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3c525-07a2-4f25-a1dc-8bd2ef74de0c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:09.506206 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 915/1877] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:09 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:09.511419 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c894ac-897f-4090-b601-662e5abb7e30 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:09.529410 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7ae3c525-07a2-4f25-a1dc-8bd2ef74de0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a54e967052b4265b99881a174443298', 'request_id': 'req-7ae3c525-07a2-4f25-a1dc-8bd2ef74de0c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:09.535054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7ae3c525-07a2-4f25-a1dc-8bd2ef74de0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a54e967052b4265b99881a174443298', 'request_id': 'req-7ae3c525-07a2-4f25-a1dc-8bd2ef74de0c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:09.541704 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7ae3c525-07a2-4f25-a1dc-8bd2ef74de0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a54e967052b4265b99881a174443298', 'request_id': 'req-7ae3c525-07a2-4f25-a1dc-8bd2ef74de0c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:09.555001 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7ae3c525-07a2-4f25-a1dc-8bd2ef74de0c admin admin] The token cache is being invalidate because project 4a54e967052b4265b99881a174443298 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:09.555289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7ae3c525-07a2-4f25-a1dc-8bd2ef74de0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:09.555926 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-7ae3c525-07a2-4f25-a1dc-8bd2ef74de0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:09.557478 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 963/1878] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:09 2022] DELETE /identity/v3/projects/4a54e967052b4265b99881a174443298 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:09.561490 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3c525-07a2-4f25-a1dc-8bd2ef74de0c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:09.562838 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7ae3c525-07a2-4f25-a1dc-8bd2ef74de0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:09.575852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c894ac-897f-4090-b601-662e5abb7e30 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:09.581216 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:09.631096 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3c525-07a2-4f25-a1dc-8bd2ef74de0c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:09.633663 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720a758-8e76-4bc6-bf4f-5ea15f7f2e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:09.634352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720a758-8e76-4bc6-bf4f-5ea15f7f2e97 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:09.634661 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720a758-8e76-4bc6-bf4f-5ea15f7f2e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:09.634954 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720a758-8e76-4bc6-bf4f-5ea15f7f2e97 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:09.635660 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e720a758-8e76-4bc6-bf4f-5ea15f7f2e97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:09.636033 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383b394-8b4a-4c36-ba60-8aac75cfe763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:09.637033 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383b394-8b4a-4c36-ba60-8aac75cfe763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:09.637326 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383b394-8b4a-4c36-ba60-8aac75cfe763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:09.637605 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383b394-8b4a-4c36-ba60-8aac75cfe763 admin admin] PATH_INFO: `/v3/users/00d9db2335b74ec3a22584b426cfcc3a` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:09.637697 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e720a758-8e76-4bc6-bf4f-5ea15f7f2e97 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:09.638843 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383b394-8b4a-4c36-ba60-8aac75cfe763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00d9db2335b74ec3a22584b426cfcc3a)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:09.640746 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383b394-8b4a-4c36-ba60-8aac75cfe763 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:09.661077 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 916/1879] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:09 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:39:09.661938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720a758-8e76-4bc6-bf4f-5ea15f7f2e97 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:09.664064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e720a758-8e76-4bc6-bf4f-5ea15f7f2e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:09.677084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720a758-8e76-4bc6-bf4f-5ea15f7f2e97 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:09.682419 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3846d7b3-4945-4c7a-a13b-c8edda56a9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:09.683109 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846d7b3-4945-4c7a-a13b-c8edda56a9d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:09.683398 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846d7b3-4945-4c7a-a13b-c8edda56a9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:09.683679 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846d7b3-4945-4c7a-a13b-c8edda56a9d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:09.684399 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3846d7b3-4945-4c7a-a13b-c8edda56a9d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:09.686053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3846d7b3-4945-4c7a-a13b-c8edda56a9d9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:09.690678 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f383b394-8b4a-4c36-ba60-8aac75cfe763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00d9db2335b74ec3a22584b426cfcc3a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:09.696504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 917/1880] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:09.700527 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f383b394-8b4a-4c36-ba60-8aac75cfe763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00d9db2335b74ec3a22584b426cfcc3a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:09.701166 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f383b394-8b4a-4c36-ba60-8aac75cfe763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00d9db2335b74ec3a22584b426cfcc3a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:09.704078 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3846d7b3-4945-4c7a-a13b-c8edda56a9d9 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:09.706382 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3846d7b3-4945-4c7a-a13b-c8edda56a9d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:09.716189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f383b394-8b4a-4c36-ba60-8aac75cfe763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00d9db2335b74ec3a22584b426cfcc3a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:09.716900 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f383b394-8b4a-4c36-ba60-8aac75cfe763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00d9db2335b74ec3a22584b426cfcc3a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:09.721346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3846d7b3-4945-4c7a-a13b-c8edda56a9d9 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:09.726704 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7f7b3-1150-44de-b1f5-daa4873ebef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:09.727414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7f7b3-1150-44de-b1f5-daa4873ebef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:09.727704 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7f7b3-1150-44de-b1f5-daa4873ebef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:09.727983 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7f7b3-1150-44de-b1f5-daa4873ebef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:09.728711 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7f7b3-1150-44de-b1f5-daa4873ebef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:09.730720 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7f7b3-1150-44de-b1f5-daa4873ebef8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:09.730889 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 964/1881] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:09 2022] DELETE /identity/v3/users/00d9db2335b74ec3a22584b426cfcc3a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:09.745167 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 918/1882] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:09.749326 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383b394-8b4a-4c36-ba60-8aac75cfe763 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:09.814345 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383b394-8b4a-4c36-ba60-8aac75cfe763 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:09.820331 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf34533-1236-4cb6-81bc-503e4602c342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:09.821223 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf34533-1236-4cb6-81bc-503e4602c342 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:09.821508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf34533-1236-4cb6-81bc-503e4602c342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:09.821785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf34533-1236-4cb6-81bc-503e4602c342 admin admin] PATH_INFO: `/v3/projects/0c0a4de948424d6ca968d4076d700d2d/users/13f0999c6db24472ba1d83c38d43f79a/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:09.836837 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf34533-1236-4cb6-81bc-503e4602c342 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c0a4de948424d6ca968d4076d700d2d, user_id=13f0999c6db24472ba1d83c38d43f79a, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:09.839754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf34533-1236-4cb6-81bc-503e4602c342 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:09.854177 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 965/1883] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:09 2022] PUT /identity/v3/projects/0c0a4de948424d6ca968d4076d700d2d/users/13f0999c6db24472ba1d83c38d43f79a/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:09.865466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18640623-cfed-4a32-b1bc-6da475fa30a4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:09.866196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18640623-cfed-4a32-b1bc-6da475fa30a4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:09.866491 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18640623-cfed-4a32-b1bc-6da475fa30a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:09.906479 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-18640623-cfed-4a32-b1bc-6da475fa30a4 None None] MFA Rules not processed for user `13f0999c6db24472ba1d83c38d43f79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:09.907961 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-18640623-cfed-4a32-b1bc-6da475fa30a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:09.934604 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:09.980865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 919/1884] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:39:09.989495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf34533-1236-4cb6-81bc-503e4602c342 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:09.991254 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-eaf34533-1236-4cb6-81bc-503e4602c342 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:10.017748 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:10.036204 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18640623-cfed-4a32-b1bc-6da475fa30a4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:10.038033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-18640623-cfed-4a32-b1bc-6da475fa30a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:10.066996 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf34533-1236-4cb6-81bc-503e4602c342 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:10.071998 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e9a91-bb47-45c1-971f-06fd63ef98a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:10.072837 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e9a91-bb47-45c1-971f-06fd63ef98a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:10.073126 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e9a91-bb47-45c1-971f-06fd63ef98a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:10.073477 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e9a91-bb47-45c1-971f-06fd63ef98a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:10.083585 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e9a91-bb47-45c1-971f-06fd63ef98a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:10.085261 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e9a91-bb47-45c1-971f-06fd63ef98a2 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:10.096495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 966/1885] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:10.110145 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18640623-cfed-4a32-b1bc-6da475fa30a4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:10.115137 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91925a8-2074-4723-a695-f2018e25406f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:10.115686 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91925a8-2074-4723-a695-f2018e25406f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:10.115969 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91925a8-2074-4723-a695-f2018e25406f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:10.116245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91925a8-2074-4723-a695-f2018e25406f admin admin] PATH_INFO: `/v3/projects/93ff53c618904fc4980d0e9459124048` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:10.117480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91925a8-2074-4723-a695-f2018e25406f admin admin] RBAC: Authorizing `identity:delete_project(project_id=93ff53c618904fc4980d0e9459124048)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:10.119411 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91925a8-2074-4723-a695-f2018e25406f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:10.167340 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f91925a8-2074-4723-a695-f2018e25406f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93ff53c618904fc4980d0e9459124048', 'request_id': 'req-f91925a8-2074-4723-a695-f2018e25406f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:10.173351 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f91925a8-2074-4723-a695-f2018e25406f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93ff53c618904fc4980d0e9459124048', 'request_id': 'req-f91925a8-2074-4723-a695-f2018e25406f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:10.179706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f91925a8-2074-4723-a695-f2018e25406f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93ff53c618904fc4980d0e9459124048', 'request_id': 'req-f91925a8-2074-4723-a695-f2018e25406f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:10.193946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f91925a8-2074-4723-a695-f2018e25406f admin admin] The token cache is being invalidate because project 93ff53c618904fc4980d0e9459124048 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:10.194210 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f91925a8-2074-4723-a695-f2018e25406f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:10.195031 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f91925a8-2074-4723-a695-f2018e25406f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:10.196621 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 920/1886] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:10 2022] DELETE /identity/v3/projects/93ff53c618904fc4980d0e9459124048 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:10.201558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b5cd8-11a4-407f-b02e-5a9d9d573037 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:10.201849 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b5cd8-11a4-407f-b02e-5a9d9d573037 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:10.202128 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b5cd8-11a4-407f-b02e-5a9d9d573037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:10.220426 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-d25b5cd8-11a4-407f-b02e-5a9d9d573037 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:10.221644 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d25b5cd8-11a4-407f-b02e-5a9d9d573037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:10.295900 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 967/1887] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:39:10.304650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3c1e9-9fdd-4664-b454-607a6d3a7635 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:10.305318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3c1e9-9fdd-4664-b454-607a6d3a7635 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:10.305673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3c1e9-9fdd-4664-b454-607a6d3a7635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:10.329495 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-00b3c1e9-9fdd-4664-b454-607a6d3a7635 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:10.330873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-00b3c1e9-9fdd-4664-b454-607a6d3a7635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:10.341275 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 921/1888] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:10.345435 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b5cd8-11a4-407f-b02e-5a9d9d573037 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:10.355953 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b5cd8-11a4-407f-b02e-5a9d9d573037 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:10.362221 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32d6cd-fee8-42d5-a3e2-53e54bc240b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:10.363558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32d6cd-fee8-42d5-a3e2-53e54bc240b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:10.363558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32d6cd-fee8-42d5-a3e2-53e54bc240b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:10.363558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32d6cd-fee8-42d5-a3e2-53e54bc240b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:10.364466 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32d6cd-fee8-42d5-a3e2-53e54bc240b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:39:10.364752 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32d6cd-fee8-42d5-a3e2-53e54bc240b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:10.366760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32d6cd-fee8-42d5-a3e2-53e54bc240b4 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:10.387324 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3c1e9-9fdd-4664-b454-607a6d3a7635 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:10.394856 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 968/1889] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:39:10.398779 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3c1e9-9fdd-4664-b454-607a6d3a7635 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:10.404438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f6af9-3fe8-4b18-9942-38e8d80bed20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:10.405123 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f6af9-3fe8-4b18-9942-38e8d80bed20 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:10.405411 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f6af9-3fe8-4b18-9942-38e8d80bed20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:10.405688 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f6af9-3fe8-4b18-9942-38e8d80bed20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:10.406390 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f6af9-3fe8-4b18-9942-38e8d80bed20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:10.410688 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f6af9-3fe8-4b18-9942-38e8d80bed20 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:10.443935 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 922/1890] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:10 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:10.448638 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32d6cd-fee8-42d5-a3e2-53e54bc240b4 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:10.459923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32d6cd-fee8-42d5-a3e2-53e54bc240b4 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:10.522654 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a62fd-fa46-4a9e-b9b0-02baeb9f2e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:10.523402 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a62fd-fa46-4a9e-b9b0-02baeb9f2e29 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:10.523702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a62fd-fa46-4a9e-b9b0-02baeb9f2e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:10.523977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a62fd-fa46-4a9e-b9b0-02baeb9f2e29 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:10.524679 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a62fd-fa46-4a9e-b9b0-02baeb9f2e29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:10.526606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a62fd-fa46-4a9e-b9b0-02baeb9f2e29 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:10.550851 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 969/1891] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:10 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:10.555029 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f6af9-3fe8-4b18-9942-38e8d80bed20 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:10.566423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f6af9-3fe8-4b18-9942-38e8d80bed20 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:10.571965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a18cb37-4df2-4faa-abd6-b3df4d49fc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:10.572125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18cb37-4df2-4faa-abd6-b3df4d49fc7c admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:10.572327 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18cb37-4df2-4faa-abd6-b3df4d49fc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:10.572703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18cb37-4df2-4faa-abd6-b3df4d49fc7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:10.573451 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a18cb37-4df2-4faa-abd6-b3df4d49fc7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:10.575736 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a18cb37-4df2-4faa-abd6-b3df4d49fc7c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:10.587337 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 923/1892] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:10.591279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a62fd-fa46-4a9e-b9b0-02baeb9f2e29 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:10.602373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a62fd-fa46-4a9e-b9b0-02baeb9f2e29 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:10.607132 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e09ca-59ea-40b8-9ec8-02811c2ef796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:10.607797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e09ca-59ea-40b8-9ec8-02811c2ef796 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:10.608076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e09ca-59ea-40b8-9ec8-02811c2ef796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:10.608480 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e09ca-59ea-40b8-9ec8-02811c2ef796 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:10.609212 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e09ca-59ea-40b8-9ec8-02811c2ef796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:10.610839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e09ca-59ea-40b8-9ec8-02811c2ef796 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:10.621981 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 970/1893] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:39:10.626776 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a18cb37-4df2-4faa-abd6-b3df4d49fc7c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:10.637723 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a18cb37-4df2-4faa-abd6-b3df4d49fc7c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:10.642567 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3833ad-424b-4860-842b-e035f33ddb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:10.643513 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3833ad-424b-4860-842b-e035f33ddb0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:10.643805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3833ad-424b-4860-842b-e035f33ddb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:10.644233 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3833ad-424b-4860-842b-e035f33ddb0a admin admin] PATH_INFO: `/v3/projects/8cff91ef8e734aa6be5bd5d0f83e5ae7/users/7fe1430e2edf4c7996c061fca4f2bd98/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:10.661432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3833ad-424b-4860-842b-e035f33ddb0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cff91ef8e734aa6be5bd5d0f83e5ae7, user_id=7fe1430e2edf4c7996c061fca4f2bd98, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:10.664767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3833ad-424b-4860-842b-e035f33ddb0a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:10.667125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:10.682594 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 924/1894] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:10 2022] PUT /identity/v3/projects/8cff91ef8e734aa6be5bd5d0f83e5ae7/users/7fe1430e2edf4c7996c061fca4f2bd98/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:10.693205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca10dc-0652-4247-af68-9b45f34ef765 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:10.693509 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca10dc-0652-4247-af68-9b45f34ef765 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:10.693858 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca10dc-0652-4247-af68-9b45f34ef765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:10.720790 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-42ca10dc-0652-4247-af68-9b45f34ef765 None None] MFA Rules not processed for user `7fe1430e2edf4c7996c061fca4f2bd98`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:10.722003 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-42ca10dc-0652-4247-af68-9b45f34ef765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:10.736502 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:10.791236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 971/1895] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:10.795711 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3833ad-424b-4860-842b-e035f33ddb0a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:10.856306 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3833ad-424b-4860-842b-e035f33ddb0a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:10.860971 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ae700-e754-43d5-aa7b-85d9456694d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:10.861636 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ae700-e754-43d5-aa7b-85d9456694d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:10.861930 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ae700-e754-43d5-aa7b-85d9456694d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:10.862212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ae700-e754-43d5-aa7b-85d9456694d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:10.862952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ae700-e754-43d5-aa7b-85d9456694d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:10.865164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ae700-e754-43d5-aa7b-85d9456694d8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:10.890615 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 925/1896] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:10 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:10.894601 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca10dc-0652-4247-af68-9b45f34ef765 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:10.905474 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca10dc-0652-4247-af68-9b45f34ef765 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:10.960226 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29971409-ceb9-4bf0-9b44-1ba76d3a6abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:10.961425 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29971409-ceb9-4bf0-9b44-1ba76d3a6abc admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:10.961707 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29971409-ceb9-4bf0-9b44-1ba76d3a6abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:10.962005 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29971409-ceb9-4bf0-9b44-1ba76d3a6abc admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:10.962714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29971409-ceb9-4bf0-9b44-1ba76d3a6abc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:10.964626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29971409-ceb9-4bf0-9b44-1ba76d3a6abc admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:10.986632 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 972/1897] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:10 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:10.990591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ae700-e754-43d5-aa7b-85d9456694d8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:11.001023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ae700-e754-43d5-aa7b-85d9456694d8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:11.005673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564c512-c5f0-41d4-976a-31ffdea3b44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:11.006345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564c512-c5f0-41d4-976a-31ffdea3b44a admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:11.006639 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564c512-c5f0-41d4-976a-31ffdea3b44a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:11.006952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564c512-c5f0-41d4-976a-31ffdea3b44a admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:11.007684 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564c512-c5f0-41d4-976a-31ffdea3b44a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:11.009282 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564c512-c5f0-41d4-976a-31ffdea3b44a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:11.020690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 926/1898] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:11.025458 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29971409-ceb9-4bf0-9b44-1ba76d3a6abc admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:11.035746 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29971409-ceb9-4bf0-9b44-1ba76d3a6abc admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:11.041212 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5599e98a-febd-4050-b619-66d5f492d132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:11.042098 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5599e98a-febd-4050-b619-66d5f492d132 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:11.042382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5599e98a-febd-4050-b619-66d5f492d132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:11.042688 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5599e98a-febd-4050-b619-66d5f492d132 admin admin] PATH_INFO: `/v3/projects/83b6de9daf1f4d2fbf64c44e88f78320/users/201d8b752c0942cd89477dc2d6b3af4d/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:11.058033 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5599e98a-febd-4050-b619-66d5f492d132 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83b6de9daf1f4d2fbf64c44e88f78320, user_id=201d8b752c0942cd89477dc2d6b3af4d, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:11.060655 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5599e98a-febd-4050-b619-66d5f492d132 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:11.075599 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 973/1899] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:11 2022] PUT /identity/v3/projects/83b6de9daf1f4d2fbf64c44e88f78320/users/201d8b752c0942cd89477dc2d6b3af4d/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:11.079584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564c512-c5f0-41d4-976a-31ffdea3b44a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:11.090351 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564c512-c5f0-41d4-976a-31ffdea3b44a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:11.154792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c35156-8166-475d-a736-57e8291f715f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:11.155526 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c35156-8166-475d-a736-57e8291f715f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:11.155814 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c35156-8166-475d-a736-57e8291f715f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:11.156294 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c35156-8166-475d-a736-57e8291f715f admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:11.157521 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c35156-8166-475d-a736-57e8291f715f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:11.159165 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c35156-8166-475d-a736-57e8291f715f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:11.172045 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 927/1900] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:11.176415 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5599e98a-febd-4050-b619-66d5f492d132 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:11.187205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5599e98a-febd-4050-b619-66d5f492d132 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:11.192063 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47f888-3fc8-4ae5-89e0-267dc826fb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:11.192963 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47f888-3fc8-4ae5-89e0-267dc826fb05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:11.193243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47f888-3fc8-4ae5-89e0-267dc826fb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:11.193516 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47f888-3fc8-4ae5-89e0-267dc826fb05 admin admin] PATH_INFO: `/v3/projects/83b6de9daf1f4d2fbf64c44e88f78320/users/201d8b752c0942cd89477dc2d6b3af4d/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:11.195492 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47f888-3fc8-4ae5-89e0-267dc826fb05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83b6de9daf1f4d2fbf64c44e88f78320, user_id=201d8b752c0942cd89477dc2d6b3af4d, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:11.198084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47f888-3fc8-4ae5-89e0-267dc826fb05 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:11.208113 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 974/1901] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:11 2022] PUT /identity/v3/projects/83b6de9daf1f4d2fbf64c44e88f78320/users/201d8b752c0942cd89477dc2d6b3af4d/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:11.218088 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1b81b-306b-4ba4-a069-3971c523600a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:11.218389 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1b81b-306b-4ba4-a069-3971c523600a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:11.218701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1b81b-306b-4ba4-a069-3971c523600a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:11.245737 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6ca1b81b-306b-4ba4-a069-3971c523600a None None] MFA Rules not processed for user `201d8b752c0942cd89477dc2d6b3af4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:11.247083 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6ca1b81b-306b-4ba4-a069-3971c523600a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:11.296292 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:11.317325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 928/1902] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:11.329269 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47f888-3fc8-4ae5-89e0-267dc826fb05 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:11.331024 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3a47f888-3fc8-4ae5-89e0-267dc826fb05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:11.387588 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47f888-3fc8-4ae5-89e0-267dc826fb05 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:11.391296 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a4029-02ac-47fa-b511-ae28ea816230 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:11.391947 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a4029-02ac-47fa-b511-ae28ea816230 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:11.392230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a4029-02ac-47fa-b511-ae28ea816230 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:11.392507 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a4029-02ac-47fa-b511-ae28ea816230 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:11.401772 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a4029-02ac-47fa-b511-ae28ea816230 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:11.403241 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a4029-02ac-47fa-b511-ae28ea816230 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:11.412610 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 975/1903] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:11.493444 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1b81b-306b-4ba4-a069-3971c523600a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:11.552338 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1b81b-306b-4ba4-a069-3971c523600a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:11.557114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9aa695-46a0-48f6-9378-3fdaad3a081d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:11.557985 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9aa695-46a0-48f6-9378-3fdaad3a081d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:11.558275 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9aa695-46a0-48f6-9378-3fdaad3a081d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:11.558556 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9aa695-46a0-48f6-9378-3fdaad3a081d admin admin] PATH_INFO: `/v3/users/7fe1430e2edf4c7996c061fca4f2bd98` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:11.559811 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9aa695-46a0-48f6-9378-3fdaad3a081d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fe1430e2edf4c7996c061fca4f2bd98)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:11.561722 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9aa695-46a0-48f6-9378-3fdaad3a081d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:11.609813 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5f9aa695-46a0-48f6-9378-3fdaad3a081d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fe1430e2edf4c7996c061fca4f2bd98'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:11.617807 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5f9aa695-46a0-48f6-9378-3fdaad3a081d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fe1430e2edf4c7996c061fca4f2bd98'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:11.624800 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5f9aa695-46a0-48f6-9378-3fdaad3a081d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fe1430e2edf4c7996c061fca4f2bd98'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:11.636773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5f9aa695-46a0-48f6-9378-3fdaad3a081d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fe1430e2edf4c7996c061fca4f2bd98'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:11.637432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5f9aa695-46a0-48f6-9378-3fdaad3a081d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fe1430e2edf4c7996c061fca4f2bd98'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:11.643422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 929/1904] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:11 2022] DELETE /identity/v3/users/7fe1430e2edf4c7996c061fca4f2bd98 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:11.648336 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034b552-d2bb-4d47-a672-9eae6776d9f1 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:11.648625 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034b552-d2bb-4d47-a672-9eae6776d9f1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:11.648899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034b552-d2bb-4d47-a672-9eae6776d9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:11.666882 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-7034b552-d2bb-4d47-a672-9eae6776d9f1 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:11.668103 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7034b552-d2bb-4d47-a672-9eae6776d9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:11.673449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:11.743069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 976/1905] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:39:11.753651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9aa695-46a0-48f6-9378-3fdaad3a081d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:11.755222 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5f9aa695-46a0-48f6-9378-3fdaad3a081d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:11.816591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9aa695-46a0-48f6-9378-3fdaad3a081d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:11.821178 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c1747-d5d9-4100-b38b-9672217cd05e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:11.821832 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c1747-d5d9-4100-b38b-9672217cd05e service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:11.822114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c1747-d5d9-4100-b38b-9672217cd05e service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:11.822388 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c1747-d5d9-4100-b38b-9672217cd05e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:11.832044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c1747-d5d9-4100-b38b-9672217cd05e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:11.833465 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c1747-d5d9-4100-b38b-9672217cd05e service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:11.844110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 930/1906] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:12.119290 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034b552-d2bb-4d47-a672-9eae6776d9f1 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:12.120925 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7034b552-d2bb-4d47-a672-9eae6776d9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:12.135030 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034b552-d2bb-4d47-a672-9eae6776d9f1 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:12.140006 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3f102-cbf0-4f0b-a079-bf269476a0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:12.142540 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3f102-cbf0-4f0b-a079-bf269476a0ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:12.142540 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3f102-cbf0-4f0b-a079-bf269476a0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:12.142540 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3f102-cbf0-4f0b-a079-bf269476a0ef admin admin] PATH_INFO: `/v3/projects/8cff91ef8e734aa6be5bd5d0f83e5ae7` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:12.144126 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d3f102-cbf0-4f0b-a079-bf269476a0ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cff91ef8e734aa6be5bd5d0f83e5ae7)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:12.145654 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d3f102-cbf0-4f0b-a079-bf269476a0ef admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:12.205398 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c7d3f102-cbf0-4f0b-a079-bf269476a0ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cff91ef8e734aa6be5bd5d0f83e5ae7', 'request_id': 'req-c7d3f102-cbf0-4f0b-a079-bf269476a0ef'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:12.213399 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c7d3f102-cbf0-4f0b-a079-bf269476a0ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cff91ef8e734aa6be5bd5d0f83e5ae7', 'request_id': 'req-c7d3f102-cbf0-4f0b-a079-bf269476a0ef'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:12.220738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c7d3f102-cbf0-4f0b-a079-bf269476a0ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cff91ef8e734aa6be5bd5d0f83e5ae7', 'request_id': 'req-c7d3f102-cbf0-4f0b-a079-bf269476a0ef'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:12.239708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c7d3f102-cbf0-4f0b-a079-bf269476a0ef admin admin] The token cache is being invalidate because project 8cff91ef8e734aa6be5bd5d0f83e5ae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:12.240086 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c7d3f102-cbf0-4f0b-a079-bf269476a0ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:12.240728 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c7d3f102-cbf0-4f0b-a079-bf269476a0ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:12.245903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 977/1907] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:12 2022] DELETE /identity/v3/projects/8cff91ef8e734aa6be5bd5d0f83e5ae7 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:12.247582 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c1747-d5d9-4100-b38b-9672217cd05e service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:12.259772 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c1747-d5d9-4100-b38b-9672217cd05e service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:12.271475 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:12.281364 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3f102-cbf0-4f0b-a079-bf269476a0ef admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:12.283283 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c7d3f102-cbf0-4f0b-a079-bf269476a0ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:12.327034 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa020379-6303-4284-8648-24e7beb5284f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:12.328731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa020379-6303-4284-8648-24e7beb5284f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:12.329160 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa020379-6303-4284-8648-24e7beb5284f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:12.329581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa020379-6303-4284-8648-24e7beb5284f admin admin] PATH_INFO: `/v3/users/201d8b752c0942cd89477dc2d6b3af4d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:12.330991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa020379-6303-4284-8648-24e7beb5284f admin admin] RBAC: Authorizing `identity:delete_user(user_id=201d8b752c0942cd89477dc2d6b3af4d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:12.333080 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa020379-6303-4284-8648-24e7beb5284f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:12.363149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3f102-cbf0-4f0b-a079-bf269476a0ef admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:12.369949 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de4ba6-843c-471c-aabc-bab220bdb0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:12.371124 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de4ba6-843c-471c-aabc-bab220bdb0d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:12.371595 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de4ba6-843c-471c-aabc-bab220bdb0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:12.372048 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de4ba6-843c-471c-aabc-bab220bdb0d6 admin admin] PATH_INFO: `/v3/users/86b04f12693c492198c01274bbc83571` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:12.373267 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de4ba6-843c-471c-aabc-bab220bdb0d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86b04f12693c492198c01274bbc83571)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:12.376888 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de4ba6-843c-471c-aabc-bab220bdb0d6 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:12.388228 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fa020379-6303-4284-8648-24e7beb5284f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '201d8b752c0942cd89477dc2d6b3af4d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:12.397356 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fa020379-6303-4284-8648-24e7beb5284f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '201d8b752c0942cd89477dc2d6b3af4d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:12.406062 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fa020379-6303-4284-8648-24e7beb5284f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '201d8b752c0942cd89477dc2d6b3af4d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:12.423449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fa020379-6303-4284-8648-24e7beb5284f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '201d8b752c0942cd89477dc2d6b3af4d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:12.424725 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fa020379-6303-4284-8648-24e7beb5284f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '201d8b752c0942cd89477dc2d6b3af4d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:12.428258 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:12.431981 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 931/1908] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:12 2022] DELETE /identity/v3/users/201d8b752c0942cd89477dc2d6b3af4d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:12.443767 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-02de4ba6-843c-471c-aabc-bab220bdb0d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86b04f12693c492198c01274bbc83571'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:12.451532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-02de4ba6-843c-471c-aabc-bab220bdb0d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86b04f12693c492198c01274bbc83571'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:12.452346 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-02de4ba6-843c-471c-aabc-bab220bdb0d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86b04f12693c492198c01274bbc83571'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:12.468073 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-02de4ba6-843c-471c-aabc-bab220bdb0d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86b04f12693c492198c01274bbc83571'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:12.469025 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-02de4ba6-843c-471c-aabc-bab220bdb0d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86b04f12693c492198c01274bbc83571'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:12.483956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 978/1909] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:12 2022] DELETE /identity/v3/users/86b04f12693c492198c01274bbc83571 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:12.489284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75ed88-4f8d-4e6e-a85f-e57eabde2437 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:12.489791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75ed88-4f8d-4e6e-a85f-e57eabde2437 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:12.491999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75ed88-4f8d-4e6e-a85f-e57eabde2437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:12.512235 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-cc75ed88-4f8d-4e6e-a85f-e57eabde2437 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:12.513601 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cc75ed88-4f8d-4e6e-a85f-e57eabde2437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:12.595627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 932/1910] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:12.610045 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de4ba6-843c-471c-aabc-bab220bdb0d6 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:12.611543 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-02de4ba6-843c-471c-aabc-bab220bdb0d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:12.677042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de4ba6-843c-471c-aabc-bab220bdb0d6 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:12.681340 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b54e34-42d6-4e7c-a1e9-b9be51b2c409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:12.682060 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b54e34-42d6-4e7c-a1e9-b9be51b2c409 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:12.682346 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b54e34-42d6-4e7c-a1e9-b9be51b2c409 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:12.682624 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b54e34-42d6-4e7c-a1e9-b9be51b2c409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:12.692919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b54e34-42d6-4e7c-a1e9-b9be51b2c409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:12.694365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b54e34-42d6-4e7c-a1e9-b9be51b2c409 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:12.704145 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 979/1911] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:12.870330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75ed88-4f8d-4e6e-a85f-e57eabde2437 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:12.871875 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cc75ed88-4f8d-4e6e-a85f-e57eabde2437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:12.885650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75ed88-4f8d-4e6e-a85f-e57eabde2437 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:12.891538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c2022-eaa2-4652-a4f3-1064a0cd9707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:12.892449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c2022-eaa2-4652-a4f3-1064a0cd9707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:12.892731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c2022-eaa2-4652-a4f3-1064a0cd9707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:12.893007 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c2022-eaa2-4652-a4f3-1064a0cd9707 admin admin] PATH_INFO: `/v3/projects/83b6de9daf1f4d2fbf64c44e88f78320` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:12.894265 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c2022-eaa2-4652-a4f3-1064a0cd9707 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83b6de9daf1f4d2fbf64c44e88f78320)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:12.896639 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c2022-eaa2-4652-a4f3-1064a0cd9707 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:12.934716 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:12.959807 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5e0c2022-eaa2-4652-a4f3-1064a0cd9707 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83b6de9daf1f4d2fbf64c44e88f78320', 'request_id': 'req-5e0c2022-eaa2-4652-a4f3-1064a0cd9707'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:12.970084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5e0c2022-eaa2-4652-a4f3-1064a0cd9707 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83b6de9daf1f4d2fbf64c44e88f78320', 'request_id': 'req-5e0c2022-eaa2-4652-a4f3-1064a0cd9707'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:12.979191 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5e0c2022-eaa2-4652-a4f3-1064a0cd9707 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83b6de9daf1f4d2fbf64c44e88f78320', 'request_id': 'req-5e0c2022-eaa2-4652-a4f3-1064a0cd9707'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:13.006256 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5e0c2022-eaa2-4652-a4f3-1064a0cd9707 admin admin] The token cache is being invalidate because project 83b6de9daf1f4d2fbf64c44e88f78320 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:13.006596 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5e0c2022-eaa2-4652-a4f3-1064a0cd9707 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:13.007401 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5e0c2022-eaa2-4652-a4f3-1064a0cd9707 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:13.013600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 933/1912] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:12 2022] DELETE /identity/v3/projects/83b6de9daf1f4d2fbf64c44e88f78320 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:13.014816 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46882e-bc7b-4e86-8f31-853a1db12d7a None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:13.015107 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46882e-bc7b-4e86-8f31-853a1db12d7a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:13.015485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46882e-bc7b-4e86-8f31-853a1db12d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:13.035630 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6b46882e-bc7b-4e86-8f31-853a1db12d7a None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:13.036863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6b46882e-bc7b-4e86-8f31-853a1db12d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:13.121496 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c2022-eaa2-4652-a4f3-1064a0cd9707 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:13.121666 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5e0c2022-eaa2-4652-a4f3-1064a0cd9707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:13.149200 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 980/1913] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:39:13.158625 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15792b26-508d-43a6-9d86-1561eab309b9 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:13.159025 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15792b26-508d-43a6-9d86-1561eab309b9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:13.159592 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15792b26-508d-43a6-9d86-1561eab309b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:13.183447 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-15792b26-508d-43a6-9d86-1561eab309b9 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:13.184675 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-15792b26-508d-43a6-9d86-1561eab309b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:13.199455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 981/1914] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:13.203665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15792b26-508d-43a6-9d86-1561eab309b9 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:13.213422 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15792b26-508d-43a6-9d86-1561eab309b9 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:13.218069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285dc090-13bf-4f6c-8630-ae555a09acbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:13.218757 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285dc090-13bf-4f6c-8630-ae555a09acbc admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:13.219047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285dc090-13bf-4f6c-8630-ae555a09acbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:13.219324 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285dc090-13bf-4f6c-8630-ae555a09acbc admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:13.220053 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285dc090-13bf-4f6c-8630-ae555a09acbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:39:13.220364 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285dc090-13bf-4f6c-8630-ae555a09acbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:13.221983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285dc090-13bf-4f6c-8630-ae555a09acbc admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:13.226542 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c2022-eaa2-4652-a4f3-1064a0cd9707 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:13.231298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01c95d-57ce-44c9-b769-ee4164babe5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:13.232179 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01c95d-57ce-44c9-b769-ee4164babe5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:13.232465 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01c95d-57ce-44c9-b769-ee4164babe5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:13.232748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01c95d-57ce-44c9-b769-ee4164babe5c admin admin] PATH_INFO: `/v3/projects/0c0895235fbc47298ff2372ce93fd24e` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:13.233938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc01c95d-57ce-44c9-b769-ee4164babe5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c0895235fbc47298ff2372ce93fd24e)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:13.235041 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 982/1915] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:39:13.235883 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc01c95d-57ce-44c9-b769-ee4164babe5c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:13.239323 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285dc090-13bf-4f6c-8630-ae555a09acbc admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:13.249106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285dc090-13bf-4f6c-8630-ae555a09acbc admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:13.253784 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a28b42-3ca8-463c-abdb-08910cdad813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:13.254419 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a28b42-3ca8-463c-abdb-08910cdad813 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:13.254732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a28b42-3ca8-463c-abdb-08910cdad813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:13.255011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a28b42-3ca8-463c-abdb-08910cdad813 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:13.255697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a28b42-3ca8-463c-abdb-08910cdad813 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:13.257607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a28b42-3ca8-463c-abdb-08910cdad813 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:13.285610 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fc01c95d-57ce-44c9-b769-ee4164babe5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c0895235fbc47298ff2372ce93fd24e', 'request_id': 'req-fc01c95d-57ce-44c9-b769-ee4164babe5c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:13.285803 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 983/1916] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:13 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:13.290413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a28b42-3ca8-463c-abdb-08910cdad813 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:13.292187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fc01c95d-57ce-44c9-b769-ee4164babe5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c0895235fbc47298ff2372ce93fd24e', 'request_id': 'req-fc01c95d-57ce-44c9-b769-ee4164babe5c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:13.298274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fc01c95d-57ce-44c9-b769-ee4164babe5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c0895235fbc47298ff2372ce93fd24e', 'request_id': 'req-fc01c95d-57ce-44c9-b769-ee4164babe5c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:13.308144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:13.311694 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fc01c95d-57ce-44c9-b769-ee4164babe5c admin admin] The token cache is being invalidate because project 0c0895235fbc47298ff2372ce93fd24e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:13.312001 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fc01c95d-57ce-44c9-b769-ee4164babe5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:13.312622 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fc01c95d-57ce-44c9-b769-ee4164babe5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:13.314257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 934/1917] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:13 2022] DELETE /identity/v3/projects/0c0895235fbc47298ff2372ce93fd24e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:13.319006 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01c95d-57ce-44c9-b769-ee4164babe5c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:13.321888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fc01c95d-57ce-44c9-b769-ee4164babe5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:13.362216 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a28b42-3ca8-463c-abdb-08910cdad813 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:13.367319 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a492d0-e1a3-4765-95fc-1a44cf2b1e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:13.368108 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a492d0-e1a3-4765-95fc-1a44cf2b1e69 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:13.368461 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a492d0-e1a3-4765-95fc-1a44cf2b1e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:13.369383 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a492d0-e1a3-4765-95fc-1a44cf2b1e69 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:13.369735 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a492d0-e1a3-4765-95fc-1a44cf2b1e69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:13.371790 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a492d0-e1a3-4765-95fc-1a44cf2b1e69 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:13.373036 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:13.393488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 984/1918] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:13 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:39:13.395154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01c95d-57ce-44c9-b769-ee4164babe5c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:13.397413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a492d0-e1a3-4765-95fc-1a44cf2b1e69 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:13.399636 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c5a492d0-e1a3-4765-95fc-1a44cf2b1e69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:13.400983 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc43fe-bc8a-4b66-82ab-a5a563fdaf96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:13.401872 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc43fe-bc8a-4b66-82ab-a5a563fdaf96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:13.402156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc43fe-bc8a-4b66-82ab-a5a563fdaf96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:13.402433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc43fe-bc8a-4b66-82ab-a5a563fdaf96 admin admin] PATH_INFO: `/v3/users/56834de3f57945ce817eb223d76511d0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:13.403655 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fc43fe-bc8a-4b66-82ab-a5a563fdaf96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56834de3f57945ce817eb223d76511d0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:13.406202 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fc43fe-bc8a-4b66-82ab-a5a563fdaf96 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:13.411233 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a492d0-e1a3-4765-95fc-1a44cf2b1e69 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:13.415868 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c5720-a7ac-47ac-9d84-856741db7e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:13.416518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c5720-a7ac-47ac-9d84-856741db7e2c admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:13.416802 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c5720-a7ac-47ac-9d84-856741db7e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:13.417075 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c5720-a7ac-47ac-9d84-856741db7e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:13.417805 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c5720-a7ac-47ac-9d84-856741db7e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:13.419417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c5720-a7ac-47ac-9d84-856741db7e2c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:13.430054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 985/1919] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:13.434041 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c5720-a7ac-47ac-9d84-856741db7e2c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:13.445378 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c5720-a7ac-47ac-9d84-856741db7e2c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:13.450338 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15adbc-7da0-444a-a5d1-672a213349c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:13.451197 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15adbc-7da0-444a-a5d1-672a213349c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:13.451296 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15adbc-7da0-444a-a5d1-672a213349c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:13.451563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15adbc-7da0-444a-a5d1-672a213349c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:13.452299 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e15adbc-7da0-444a-a5d1-672a213349c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:13.453884 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e15adbc-7da0-444a-a5d1-672a213349c2 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:13.457648 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-14fc43fe-bc8a-4b66-82ab-a5a563fdaf96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56834de3f57945ce817eb223d76511d0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:13.465756 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-14fc43fe-bc8a-4b66-82ab-a5a563fdaf96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56834de3f57945ce817eb223d76511d0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:13.467581 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 986/1920] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:39:13.472321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15adbc-7da0-444a-a5d1-672a213349c2 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:13.474058 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-14fc43fe-bc8a-4b66-82ab-a5a563fdaf96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56834de3f57945ce817eb223d76511d0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:13.486862 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15adbc-7da0-444a-a5d1-672a213349c2 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:13.487858 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-14fc43fe-bc8a-4b66-82ab-a5a563fdaf96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56834de3f57945ce817eb223d76511d0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:13.488715 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-14fc43fe-bc8a-4b66-82ab-a5a563fdaf96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56834de3f57945ce817eb223d76511d0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:13.494949 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37dcecd-1a27-4c20-b22e-4073415bcbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:13.495571 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 935/1921] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:13 2022] DELETE /identity/v3/users/56834de3f57945ce817eb223d76511d0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:13.496008 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37dcecd-1a27-4c20-b22e-4073415bcbe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:13.496365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37dcecd-1a27-4c20-b22e-4073415bcbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:13.496714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37dcecd-1a27-4c20-b22e-4073415bcbe5 admin admin] PATH_INFO: `/v3/projects/823dee98305b47c1b4c1f42ddb64b5ed/users/68b2083a3c1343d690f632f6a1a42897/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:13.513588 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37dcecd-1a27-4c20-b22e-4073415bcbe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=823dee98305b47c1b4c1f42ddb64b5ed, user_id=68b2083a3c1343d690f632f6a1a42897, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:13.516462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37dcecd-1a27-4c20-b22e-4073415bcbe5 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:13.543430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 987/1922] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:13 2022] PUT /identity/v3/projects/823dee98305b47c1b4c1f42ddb64b5ed/users/68b2083a3c1343d690f632f6a1a42897/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:13.556102 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50275e-a11b-44f1-bd83-d6ca39aee8b9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:13.556270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50275e-a11b-44f1-bd83-d6ca39aee8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:13.556536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50275e-a11b-44f1-bd83-d6ca39aee8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:13.604122 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1b50275e-a11b-44f1-bd83-d6ca39aee8b9 None None] MFA Rules not processed for user `68b2083a3c1343d690f632f6a1a42897`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:13.606315 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1b50275e-a11b-44f1-bd83-d6ca39aee8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:13.725458 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 936/1923] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:13.730811 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37dcecd-1a27-4c20-b22e-4073415bcbe5 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:13.732769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f37dcecd-1a27-4c20-b22e-4073415bcbe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:13.816795 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37dcecd-1a27-4c20-b22e-4073415bcbe5 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:13.822555 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f546a1-c886-4b90-afc8-7c04e68dc9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:13.822555 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f546a1-c886-4b90-afc8-7c04e68dc9bb admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:13.822555 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f546a1-c886-4b90-afc8-7c04e68dc9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:13.822555 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f546a1-c886-4b90-afc8-7c04e68dc9bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:13.824659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f546a1-c886-4b90-afc8-7c04e68dc9bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:13.825731 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f546a1-c886-4b90-afc8-7c04e68dc9bb admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:13.851590 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b50275e-a11b-44f1-bd83-d6ca39aee8b9 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:13.854338 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1b50275e-a11b-44f1-bd83-d6ca39aee8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:13.876813 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 988/1924] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:13 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:13.886301 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f546a1-c886-4b90-afc8-7c04e68dc9bb admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:13.892584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b50275e-a11b-44f1-bd83-d6ca39aee8b9 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:13.968080 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:13.974776 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:13.989002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe7e01-077d-4fab-8ad1-391c855f3dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:13.990010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe7e01-077d-4fab-8ad1-391c855f3dbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:13.990406 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe7e01-077d-4fab-8ad1-391c855f3dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:13.990680 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f546a1-c886-4b90-afc8-7c04e68dc9bb admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:13.990788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe7e01-077d-4fab-8ad1-391c855f3dbd admin admin] PATH_INFO: `/v3/projects/f653d405e6ef4e048536717aad8591a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:13.992957 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe7e01-077d-4fab-8ad1-391c855f3dbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f653d405e6ef4e048536717aad8591a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:13.995861 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe7e01-077d-4fab-8ad1-391c855f3dbd admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:13.996468 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea218a1b-c159-4c50-9c45-315e1c7d34a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:13.997144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea218a1b-c159-4c50-9c45-315e1c7d34a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:13.997428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea218a1b-c159-4c50-9c45-315e1c7d34a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:13.997707 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea218a1b-c159-4c50-9c45-315e1c7d34a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:13.998597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea218a1b-c159-4c50-9c45-315e1c7d34a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:14.000930 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea218a1b-c159-4c50-9c45-315e1c7d34a9 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:14.040214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 989/1925] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:13 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:14.041773 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea218a1b-c159-4c50-9c45-315e1c7d34a9 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:14.058566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea218a1b-c159-4c50-9c45-315e1c7d34a9 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:14.071852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dfbe7e01-077d-4fab-8ad1-391c855f3dbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f653d405e6ef4e048536717aad8591a1', 'request_id': 'req-dfbe7e01-077d-4fab-8ad1-391c855f3dbd'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:14.084370 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dfbe7e01-077d-4fab-8ad1-391c855f3dbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f653d405e6ef4e048536717aad8591a1', 'request_id': 'req-dfbe7e01-077d-4fab-8ad1-391c855f3dbd'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:14.091574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dfbe7e01-077d-4fab-8ad1-391c855f3dbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f653d405e6ef4e048536717aad8591a1', 'request_id': 'req-dfbe7e01-077d-4fab-8ad1-391c855f3dbd'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:14.117515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dfbe7e01-077d-4fab-8ad1-391c855f3dbd admin admin] The token cache is being invalidate because project f653d405e6ef4e048536717aad8591a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:14.121481 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dfbe7e01-077d-4fab-8ad1-391c855f3dbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:14.121481 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dfbe7e01-077d-4fab-8ad1-391c855f3dbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:14.121481 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 937/1926] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:13 2022] DELETE /identity/v3/projects/f653d405e6ef4e048536717aad8591a1 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:14.127174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95062884-c0ba-49bd-8175-f3a1a5f578f7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:14.127512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95062884-c0ba-49bd-8175-f3a1a5f578f7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:14.127830 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95062884-c0ba-49bd-8175-f3a1a5f578f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:14.139330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e76875-1763-4056-9192-21b6f87b80bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:14.140088 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e76875-1763-4056-9192-21b6f87b80bc admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:14.140467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e76875-1763-4056-9192-21b6f87b80bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:14.140817 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e76875-1763-4056-9192-21b6f87b80bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:14.141669 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e76875-1763-4056-9192-21b6f87b80bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:14.143470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e76875-1763-4056-9192-21b6f87b80bc admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:14.150007 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-95062884-c0ba-49bd-8175-f3a1a5f578f7 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:14.151291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-95062884-c0ba-49bd-8175-f3a1a5f578f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:14.157533 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 990/1927] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:14.158761 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e76875-1763-4056-9192-21b6f87b80bc admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:14.160655 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c5e76875-1763-4056-9192-21b6f87b80bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:14.162678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 938/1928] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:14.172206 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4070d8-bb65-42af-a4e9-556a3106bc0a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:14.172621 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4070d8-bb65-42af-a4e9-556a3106bc0a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:14.172969 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4070d8-bb65-42af-a4e9-556a3106bc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:14.178886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e76875-1763-4056-9192-21b6f87b80bc admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:14.185298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a525ca-77fd-459b-b875-4d759bc703e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:14.186000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a525ca-77fd-459b-b875-4d759bc703e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:14.186332 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a525ca-77fd-459b-b875-4d759bc703e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:14.186593 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a525ca-77fd-459b-b875-4d759bc703e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:14.187384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a525ca-77fd-459b-b875-4d759bc703e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:14.189141 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a525ca-77fd-459b-b875-4d759bc703e0 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:14.196718 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1f4070d8-bb65-42af-a4e9-556a3106bc0a None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:14.197953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1f4070d8-bb65-42af-a4e9-556a3106bc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:14.202715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 991/1929] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:39:14.204386 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a525ca-77fd-459b-b875-4d759bc703e0 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:14.206866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 939/1930] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:14.210907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4070d8-bb65-42af-a4e9-556a3106bc0a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:14.224449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a525ca-77fd-459b-b875-4d759bc703e0 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:14.231603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8876c-31f1-4fda-81cc-86b9f948c7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:14.232683 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8876c-31f1-4fda-81cc-86b9f948c7e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:14.233126 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8876c-31f1-4fda-81cc-86b9f948c7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:14.234788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4070d8-bb65-42af-a4e9-556a3106bc0a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:14.238276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8876c-31f1-4fda-81cc-86b9f948c7e6 admin admin] PATH_INFO: `/v3/projects/ef0be99ab21c43219b09225daefd7300/users/d8c0f3fbf1e84c1b8a7fc8fb5225c8b4/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:14.247769 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfb896-c030-4906-9c5c-1c9e9469c248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:14.249255 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfb896-c030-4906-9c5c-1c9e9469c248 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:14.249629 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfb896-c030-4906-9c5c-1c9e9469c248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:14.249963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfb896-c030-4906-9c5c-1c9e9469c248 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:14.251009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbfb896-c030-4906-9c5c-1c9e9469c248 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:39:14.251355 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbfb896-c030-4906-9c5c-1c9e9469c248 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:14.252973 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbfb896-c030-4906-9c5c-1c9e9469c248 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:14.268573 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8876c-31f1-4fda-81cc-86b9f948c7e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef0be99ab21c43219b09225daefd7300, user_id=d8c0f3fbf1e84c1b8a7fc8fb5225c8b4, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:14.269624 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 940/1931] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:39:14.270677 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfb896-c030-4906-9c5c-1c9e9469c248 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:14.273960 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8876c-31f1-4fda-81cc-86b9f948c7e6 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:14.281937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfb896-c030-4906-9c5c-1c9e9469c248 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:14.287611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b51fe-e63e-4256-adb9-4d85aae0dcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:14.289071 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b51fe-e63e-4256-adb9-4d85aae0dcbb admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:14.289329 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b51fe-e63e-4256-adb9-4d85aae0dcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:14.289546 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b51fe-e63e-4256-adb9-4d85aae0dcbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:14.290254 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b51fe-e63e-4256-adb9-4d85aae0dcbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:14.292517 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b51fe-e63e-4256-adb9-4d85aae0dcbb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:14.298685 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 992/1932] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:14 2022] PUT /identity/v3/projects/ef0be99ab21c43219b09225daefd7300/users/d8c0f3fbf1e84c1b8a7fc8fb5225c8b4/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:14.309615 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d9f01-36b8-4605-938d-01495a4dc192 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:14.309904 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d9f01-36b8-4605-938d-01495a4dc192 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:14.310191 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d9f01-36b8-4605-938d-01495a4dc192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:14.327126 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 941/1933] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:14 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:14.331225 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b51fe-e63e-4256-adb9-4d85aae0dcbb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:14.346684 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-d58d9f01-36b8-4605-938d-01495a4dc192 None None] MFA Rules not processed for user `d8c0f3fbf1e84c1b8a7fc8fb5225c8b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:14.347632 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d58d9f01-36b8-4605-938d-01495a4dc192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:14.429701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b51fe-e63e-4256-adb9-4d85aae0dcbb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:14.436710 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 993/1934] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:39:14.436999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192ac3f-63e4-4787-88cc-dcb144037550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:14.437723 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192ac3f-63e4-4787-88cc-dcb144037550 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:14.438012 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192ac3f-63e4-4787-88cc-dcb144037550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:14.438404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192ac3f-63e4-4787-88cc-dcb144037550 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:14.439430 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192ac3f-63e4-4787-88cc-dcb144037550 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:14.440405 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d9f01-36b8-4605-938d-01495a4dc192 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:14.441770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192ac3f-63e4-4787-88cc-dcb144037550 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:14.458113 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d9f01-36b8-4605-938d-01495a4dc192 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:14.465328 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2969109d-20af-403b-b990-66bc43711b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:14.466030 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969109d-20af-403b-b990-66bc43711b25 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:14.466355 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969109d-20af-403b-b990-66bc43711b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:14.467309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969109d-20af-403b-b990-66bc43711b25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:14.471374 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2969109d-20af-403b-b990-66bc43711b25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:14.473505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 942/1935] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:14 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:39:14.473663 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2969109d-20af-403b-b990-66bc43711b25 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:14.477749 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192ac3f-63e4-4787-88cc-dcb144037550 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:14.491639 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192ac3f-63e4-4787-88cc-dcb144037550 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:14.498444 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be058bb-749f-4f89-8266-39c8a742dfb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:14.499763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be058bb-749f-4f89-8266-39c8a742dfb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:14.499763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be058bb-749f-4f89-8266-39c8a742dfb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:14.499961 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be058bb-749f-4f89-8266-39c8a742dfb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:14.500636 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be058bb-749f-4f89-8266-39c8a742dfb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:14.502380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be058bb-749f-4f89-8266-39c8a742dfb5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:14.514729 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 943/1936] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:39:14.517792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be058bb-749f-4f89-8266-39c8a742dfb5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:14.523317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 994/1937] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:14 2022] POST /identity/v3/projects => generated 385 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:39:14.527610 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2969109d-20af-403b-b990-66bc43711b25 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:14.529984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be058bb-749f-4f89-8266-39c8a742dfb5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:14.535574 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:14.544379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:14.600188 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba022e41-c9bc-49f8-a3b0-fcd3701f06dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:14.601739 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba022e41-c9bc-49f8-a3b0-fcd3701f06dd admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:14.602060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba022e41-c9bc-49f8-a3b0-fcd3701f06dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:14.602337 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba022e41-c9bc-49f8-a3b0-fcd3701f06dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:14.603118 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba022e41-c9bc-49f8-a3b0-fcd3701f06dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:14.604710 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba022e41-c9bc-49f8-a3b0-fcd3701f06dd admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:14.605250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2969109d-20af-403b-b990-66bc43711b25 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:14.610063 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54137b-a12b-41fc-9f6e-2e666f54c380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:14.610853 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54137b-a12b-41fc-9f6e-2e666f54c380 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:14.611205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54137b-a12b-41fc-9f6e-2e666f54c380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:14.611548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54137b-a12b-41fc-9f6e-2e666f54c380 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:14.612358 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54137b-a12b-41fc-9f6e-2e666f54c380 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:14.614350 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54137b-a12b-41fc-9f6e-2e666f54c380 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:14.616049 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 944/1938] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:39:14.620021 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba022e41-c9bc-49f8-a3b0-fcd3701f06dd admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:14.631656 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba022e41-c9bc-49f8-a3b0-fcd3701f06dd admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:14.636567 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5993d-bbd2-4310-874b-147d707a5e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:14.637568 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5993d-bbd2-4310-874b-147d707a5e6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:14.637732 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5993d-bbd2-4310-874b-147d707a5e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:14.638011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5993d-bbd2-4310-874b-147d707a5e6c admin admin] PATH_INFO: `/v3/projects/135d9c578ad04be69c691d4b43ad507c/users/1307f4f9c84f4568a84f10142df9f3d8/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:14.640702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 995/1939] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:14 2022] POST /identity/v3/users => generated 299 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:39:14.644451 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54137b-a12b-41fc-9f6e-2e666f54c380 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:14.655823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c5993d-bbd2-4310-874b-147d707a5e6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=135d9c578ad04be69c691d4b43ad507c, user_id=1307f4f9c84f4568a84f10142df9f3d8, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:14.656391 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54137b-a12b-41fc-9f6e-2e666f54c380 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:14.658838 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c5993d-bbd2-4310-874b-147d707a5e6c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:14.661276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a2c86-ccf4-452d-8303-e1e8d2c64ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:14.661948 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a2c86-ccf4-452d-8303-e1e8d2c64ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:14.662239 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a2c86-ccf4-452d-8303-e1e8d2c64ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:14.662513 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a2c86-ccf4-452d-8303-e1e8d2c64ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:14.663422 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a2c86-ccf4-452d-8303-e1e8d2c64ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:14.665781 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a2c86-ccf4-452d-8303-e1e8d2c64ce6 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:14.675155 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 945/1940] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:14 2022] PUT /identity/v3/projects/135d9c578ad04be69c691d4b43ad507c/users/1307f4f9c84f4568a84f10142df9f3d8/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:14.681799 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 996/1941] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:14.686257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5993d-bbd2-4310-874b-147d707a5e6c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:14.687566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5448cd6-355c-4434-a8f6-840bf6bd22a4 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:14.687863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5448cd6-355c-4434-a8f6-840bf6bd22a4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:14.688148 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5448cd6-355c-4434-a8f6-840bf6bd22a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:14.719148 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-e5448cd6-355c-4434-a8f6-840bf6bd22a4 None None] MFA Rules not processed for user `1307f4f9c84f4568a84f10142df9f3d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:14.720590 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e5448cd6-355c-4434-a8f6-840bf6bd22a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:14.770053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5993d-bbd2-4310-874b-147d707a5e6c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:14.775468 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1915d-6500-443a-83e4-7526a34d7f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:14.776375 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1915d-6500-443a-83e4-7526a34d7f23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:14.776659 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1915d-6500-443a-83e4-7526a34d7f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:14.776942 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1915d-6500-443a-83e4-7526a34d7f23 admin admin] PATH_INFO: `/v3/projects/7a7b16d2fe324bf3ae9afbd120e3e4e3/users/99c9b3aa5fcc484dbfa0e838c84cc7e4/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:14.791912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 997/1942] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:14.792078 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1915d-6500-443a-83e4-7526a34d7f23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a7b16d2fe324bf3ae9afbd120e3e4e3, user_id=99c9b3aa5fcc484dbfa0e838c84cc7e4, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:14.794622 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1915d-6500-443a-83e4-7526a34d7f23 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:14.796274 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5448cd6-355c-4434-a8f6-840bf6bd22a4 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:14.808277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5448cd6-355c-4434-a8f6-840bf6bd22a4 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:14.810046 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 946/1943] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:14 2022] PUT /identity/v3/projects/7a7b16d2fe324bf3ae9afbd120e3e4e3/users/99c9b3aa5fcc484dbfa0e838c84cc7e4/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:14.814175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1915d-6500-443a-83e4-7526a34d7f23 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:14.864332 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52425f86-8cf9-4095-9ab2-15fd8751f1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:14.865966 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52425f86-8cf9-4095-9ab2-15fd8751f1fc admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:14.865966 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52425f86-8cf9-4095-9ab2-15fd8751f1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:14.865966 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52425f86-8cf9-4095-9ab2-15fd8751f1fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:14.867121 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52425f86-8cf9-4095-9ab2-15fd8751f1fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:14.868549 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52425f86-8cf9-4095-9ab2-15fd8751f1fc admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:14.880043 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1915d-6500-443a-83e4-7526a34d7f23 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:14.883863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:14.885959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49f9f5-b851-490b-b650-669eb7f23ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:14.886763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49f9f5-b851-490b-b650-669eb7f23ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:14.887087 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49f9f5-b851-490b-b650-669eb7f23ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:14.887451 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49f9f5-b851-490b-b650-669eb7f23ea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:14.888236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49f9f5-b851-490b-b650-669eb7f23ea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:14.889994 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49f9f5-b851-490b-b650-669eb7f23ea6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:14.900535 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 998/1944] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:14 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:39:14.902694 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 947/1945] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:39:14.904806 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52425f86-8cf9-4095-9ab2-15fd8751f1fc admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:14.906305 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49f9f5-b851-490b-b650-669eb7f23ea6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:14.913488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:14.919453 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49f9f5-b851-490b-b650-669eb7f23ea6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:14.974680 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52425f86-8cf9-4095-9ab2-15fd8751f1fc admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:14.980363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b620e7da-17cc-4c6e-8a19-70eee8cd26b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:14.981046 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620e7da-17cc-4c6e-8a19-70eee8cd26b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:14.981316 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620e7da-17cc-4c6e-8a19-70eee8cd26b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:14.981623 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620e7da-17cc-4c6e-8a19-70eee8cd26b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:14.982391 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b620e7da-17cc-4c6e-8a19-70eee8cd26b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:14.982609 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593950ba-0a5a-47f6-995a-1733fa36bde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:14.983880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593950ba-0a5a-47f6-995a-1733fa36bde0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:14.984229 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593950ba-0a5a-47f6-995a-1733fa36bde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:14.984587 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593950ba-0a5a-47f6-995a-1733fa36bde0 admin admin] PATH_INFO: `/v3/projects/7a7b16d2fe324bf3ae9afbd120e3e4e3/users/99c9b3aa5fcc484dbfa0e838c84cc7e4/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:14.984849 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b620e7da-17cc-4c6e-8a19-70eee8cd26b1 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:14.987070 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593950ba-0a5a-47f6-995a-1733fa36bde0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a7b16d2fe324bf3ae9afbd120e3e4e3, user_id=99c9b3aa5fcc484dbfa0e838c84cc7e4, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:14.989671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593950ba-0a5a-47f6-995a-1733fa36bde0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:15.000874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 948/1946] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:14 2022] PUT /identity/v3/projects/7a7b16d2fe324bf3ae9afbd120e3e4e3/users/99c9b3aa5fcc484dbfa0e838c84cc7e4/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:15.009626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 999/1947] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:14 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:39:15.010875 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593950ba-0a5a-47f6-995a-1733fa36bde0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:15.012308 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898bd2c-1ede-420b-ab8c-47e26982b88c None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:15.012603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898bd2c-1ede-420b-ab8c-47e26982b88c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:15.012886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898bd2c-1ede-420b-ab8c-47e26982b88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:15.039814 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-1898bd2c-1ede-420b-ab8c-47e26982b88c None None] MFA Rules not processed for user `99c9b3aa5fcc484dbfa0e838c84cc7e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:15.041051 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1898bd2c-1ede-420b-ab8c-47e26982b88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:15.075332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593950ba-0a5a-47f6-995a-1733fa36bde0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:15.080044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf91632-b651-4731-8a5f-9fe28b206c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:15.080715 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf91632-b651-4731-8a5f-9fe28b206c91 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:15.081008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf91632-b651-4731-8a5f-9fe28b206c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:15.081292 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf91632-b651-4731-8a5f-9fe28b206c91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:15.082017 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf91632-b651-4731-8a5f-9fe28b206c91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:15.083648 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf91632-b651-4731-8a5f-9fe28b206c91 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:15.094102 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 949/1948] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:39:15.098124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf91632-b651-4731-8a5f-9fe28b206c91 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:15.107446 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf91632-b651-4731-8a5f-9fe28b206c91 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:15.111539 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1000/1949] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:39:15.112463 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141d018-ba79-4457-9516-06c3a9848fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:15.113341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141d018-ba79-4457-9516-06c3a9848fd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:15.113629 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141d018-ba79-4457-9516-06c3a9848fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:15.113907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141d018-ba79-4457-9516-06c3a9848fd2 admin admin] PATH_INFO: `/v3/projects/9fa1f1afd23147e5a73e3ddb055213a8/users/3eb7c87d5e6f4c89964a15d552010e73/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:15.128376 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898bd2c-1ede-420b-ab8c-47e26982b88c None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:15.129802 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1898bd2c-1ede-420b-ab8c-47e26982b88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:15.137569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a141d018-ba79-4457-9516-06c3a9848fd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fa1f1afd23147e5a73e3ddb055213a8, user_id=3eb7c87d5e6f4c89964a15d552010e73, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:15.140365 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a141d018-ba79-4457-9516-06c3a9848fd2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:15.161763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 950/1950] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:15 2022] PUT /identity/v3/projects/9fa1f1afd23147e5a73e3ddb055213a8/users/3eb7c87d5e6f4c89964a15d552010e73/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:15.162913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141d018-ba79-4457-9516-06c3a9848fd2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:15.211294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898bd2c-1ede-420b-ab8c-47e26982b88c None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:15.237904 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:15.239304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141d018-ba79-4457-9516-06c3a9848fd2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:15.246500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cecb62-03d4-4667-8416-a937709ad536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:15.247328 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cecb62-03d4-4667-8416-a937709ad536 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:15.247626 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cecb62-03d4-4667-8416-a937709ad536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:15.248122 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cecb62-03d4-4667-8416-a937709ad536 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:15.248954 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cecb62-03d4-4667-8416-a937709ad536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:15.250772 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cecb62-03d4-4667-8416-a937709ad536 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:15.263106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 951/1951] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:15.266873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cecb62-03d4-4667-8416-a937709ad536 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:15.275063 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41649fa2-0e58-41cf-94da-90edcd72cacb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:15.275804 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41649fa2-0e58-41cf-94da-90edcd72cacb service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:15.276158 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41649fa2-0e58-41cf-94da-90edcd72cacb service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:15.276502 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41649fa2-0e58-41cf-94da-90edcd72cacb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:15.277548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cecb62-03d4-4667-8416-a937709ad536 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:15.282179 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd7beb3-4b13-4829-a42e-b53df3701b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:15.283122 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd7beb3-4b13-4829-a42e-b53df3701b1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:15.283434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd7beb3-4b13-4829-a42e-b53df3701b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:15.283712 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd7beb3-4b13-4829-a42e-b53df3701b1d admin admin] PATH_INFO: `/v3/projects/9fa1f1afd23147e5a73e3ddb055213a8/users/3eb7c87d5e6f4c89964a15d552010e73/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:15.285683 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd7beb3-4b13-4829-a42e-b53df3701b1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fa1f1afd23147e5a73e3ddb055213a8, user_id=3eb7c87d5e6f4c89964a15d552010e73, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:15.288361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd7beb3-4b13-4829-a42e-b53df3701b1d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:15.290846 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:15.300976 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 952/1952] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:15 2022] PUT /identity/v3/projects/9fa1f1afd23147e5a73e3ddb055213a8/users/3eb7c87d5e6f4c89964a15d552010e73/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:15.311585 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d25eac-5ce6-4a38-8d89-48c0ec40103a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:15.311883 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d25eac-5ce6-4a38-8d89-48c0ec40103a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:15.312168 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d25eac-5ce6-4a38-8d89-48c0ec40103a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:15.339614 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d7d25eac-5ce6-4a38-8d89-48c0ec40103a None None] MFA Rules not processed for user `3eb7c87d5e6f4c89964a15d552010e73`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:15.340823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d7d25eac-5ce6-4a38-8d89-48c0ec40103a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:15.352460 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41649fa2-0e58-41cf-94da-90edcd72cacb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:15.353901 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41649fa2-0e58-41cf-94da-90edcd72cacb service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:15.363407 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1001/1953] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:15.421877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 953/1954] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:15 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:39:15.996047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41649fa2-0e58-41cf-94da-90edcd72cacb service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:16.004446 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41649fa2-0e58-41cf-94da-90edcd72cacb service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:16.051519 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1d2e2-fa63-4f29-817c-837729ffed91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:16.052100 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1d2e2-fa63-4f29-817c-837729ffed91 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:16.052384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1d2e2-fa63-4f29-817c-837729ffed91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:16.052662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1d2e2-fa63-4f29-817c-837729ffed91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:16.054563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b6e1d2e2-fa63-4f29-817c-837729ffed91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:16.117708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1d2e2-fa63-4f29-817c-837729ffed91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:16.119337 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1d2e2-fa63-4f29-817c-837729ffed91 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:16.129092 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1002/1955] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:22.726031 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf507f-ee62-4579-8b01-a5bdf10953c7 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:22.726548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf507f-ee62-4579-8b01-a5bdf10953c7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:22.726669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf507f-ee62-4579-8b01-a5bdf10953c7 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:22.728020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 954/1956] 10.222.0.67 () {54 vars in 921 bytes} [Tue May 31 14:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:39:22.740144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfff41-ea3d-4956-ac45-c40d505f058b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:22.740632 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfff41-ea3d-4956-ac45-c40d505f058b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:22.740746 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfff41-ea3d-4956-ac45-c40d505f058b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:22.761412 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-c4bfff41-ea3d-4956-ac45-c40d505f058b None None] MFA Rules not processed for user `68c6363b60954c3ebf50d2019f5f2fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:22.762632 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c4bfff41-ea3d-4956-ac45-c40d505f058b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:22.790185 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:22.836985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1003/1957] 10.222.0.67 () {58 vars in 1018 bytes} [Tue May 31 14:39:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:22.849221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf507f-ee62-4579-8b01-a5bdf10953c7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:22.850953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-40bf507f-ee62-4579-8b01-a5bdf10953c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:22.868342 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf507f-ee62-4579-8b01-a5bdf10953c7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:22.874307 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770f86a-f71b-4ef1-ba3f-091d62e0274f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:22.874988 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770f86a-f71b-4ef1-ba3f-091d62e0274f service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:22.875274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770f86a-f71b-4ef1-ba3f-091d62e0274f service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:22.875550 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770f86a-f71b-4ef1-ba3f-091d62e0274f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:22.884275 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0770f86a-f71b-4ef1-ba3f-091d62e0274f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:22.886459 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0770f86a-f71b-4ef1-ba3f-091d62e0274f service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:22.896510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 955/1958] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:23.391447 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfff41-ea3d-4956-ac45-c40d505f058b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:23.393101 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c4bfff41-ea3d-4956-ac45-c40d505f058b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:23.498124 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfff41-ea3d-4956-ac45-c40d505f058b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:23.508280 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c53f05-755b-4515-99ba-2eb64c457548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:23.508280 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c53f05-755b-4515-99ba-2eb64c457548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:23.508280 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c53f05-755b-4515-99ba-2eb64c457548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:23.508280 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c53f05-755b-4515-99ba-2eb64c457548 admin admin] PATH_INFO: `/v3/users/4940be82d6074beb9c0b40895e87bcad` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:23.510096 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c53f05-755b-4515-99ba-2eb64c457548 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4940be82d6074beb9c0b40895e87bcad)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:23.513253 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c53f05-755b-4515-99ba-2eb64c457548 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:23.590606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-34c53f05-755b-4515-99ba-2eb64c457548 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4940be82d6074beb9c0b40895e87bcad'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:23.603868 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-34c53f05-755b-4515-99ba-2eb64c457548 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4940be82d6074beb9c0b40895e87bcad'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:23.604996 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-34c53f05-755b-4515-99ba-2eb64c457548 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4940be82d6074beb9c0b40895e87bcad'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:23.621641 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-34c53f05-755b-4515-99ba-2eb64c457548 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4940be82d6074beb9c0b40895e87bcad'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:23.622581 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-34c53f05-755b-4515-99ba-2eb64c457548 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4940be82d6074beb9c0b40895e87bcad'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:23.640392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1004/1959] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:23 2022] DELETE /identity/v3/users/4940be82d6074beb9c0b40895e87bcad => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:23.644944 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa90ed8-40ce-4e1b-a0b0-8700a9e14fe1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:23.645260 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa90ed8-40ce-4e1b-a0b0-8700a9e14fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:23.645556 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa90ed8-40ce-4e1b-a0b0-8700a9e14fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:23.665234 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6fa90ed8-40ce-4e1b-a0b0-8700a9e14fe1 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:23.666481 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6fa90ed8-40ce-4e1b-a0b0-8700a9e14fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:23.740635 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 956/1960] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:23.751756 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c53f05-755b-4515-99ba-2eb64c457548 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:23.754309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-34c53f05-755b-4515-99ba-2eb64c457548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:23.825922 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:23.856878 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c53f05-755b-4515-99ba-2eb64c457548 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:23.861090 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2beda8a-207c-466a-a6e1-3f11ed331b44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:23.861751 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2beda8a-207c-466a-a6e1-3f11ed331b44 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:23.862037 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2beda8a-207c-466a-a6e1-3f11ed331b44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:23.862313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2beda8a-207c-466a-a6e1-3f11ed331b44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:23.873835 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2beda8a-207c-466a-a6e1-3f11ed331b44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:23.875680 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2beda8a-207c-466a-a6e1-3f11ed331b44 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:23.894460 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1005/1961] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:39:24.226424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa90ed8-40ce-4e1b-a0b0-8700a9e14fe1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:24.229403 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6fa90ed8-40ce-4e1b-a0b0-8700a9e14fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:24.248416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa90ed8-40ce-4e1b-a0b0-8700a9e14fe1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:24.256170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7127e4b-d8b3-4fd6-a689-b981fdbc893a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:24.257268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7127e4b-d8b3-4fd6-a689-b981fdbc893a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:24.257658 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7127e4b-d8b3-4fd6-a689-b981fdbc893a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:24.258054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7127e4b-d8b3-4fd6-a689-b981fdbc893a admin admin] PATH_INFO: `/v3/projects/fdd0e28e9b7a4e019ae3804d5bc3424f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:24.260823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7127e4b-d8b3-4fd6-a689-b981fdbc893a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdd0e28e9b7a4e019ae3804d5bc3424f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:24.264255 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7127e4b-d8b3-4fd6-a689-b981fdbc893a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:24.316938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a7127e4b-d8b3-4fd6-a689-b981fdbc893a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdd0e28e9b7a4e019ae3804d5bc3424f', 'request_id': 'req-a7127e4b-d8b3-4fd6-a689-b981fdbc893a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:24.324720 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a7127e4b-d8b3-4fd6-a689-b981fdbc893a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdd0e28e9b7a4e019ae3804d5bc3424f', 'request_id': 'req-a7127e4b-d8b3-4fd6-a689-b981fdbc893a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:24.326623 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:INSERT=1,keystone:SELECT=93,keystone:DELETE=6 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:24.331598 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a7127e4b-d8b3-4fd6-a689-b981fdbc893a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdd0e28e9b7a4e019ae3804d5bc3424f', 'request_id': 'req-a7127e4b-d8b3-4fd6-a689-b981fdbc893a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:24.344974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a7127e4b-d8b3-4fd6-a689-b981fdbc893a admin admin] The token cache is being invalidate because project fdd0e28e9b7a4e019ae3804d5bc3424f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:24.345319 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a7127e4b-d8b3-4fd6-a689-b981fdbc893a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:24.346051 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a7127e4b-d8b3-4fd6-a689-b981fdbc893a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:24.347917 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 957/1962] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:24 2022] DELETE /identity/v3/projects/fdd0e28e9b7a4e019ae3804d5bc3424f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:24.352512 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2beda8a-207c-466a-a6e1-3f11ed331b44 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:24.362293 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2beda8a-207c-466a-a6e1-3f11ed331b44 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:24.422869 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49167a55-c33b-401b-aba0-02bdbdcdaefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:24.423858 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49167a55-c33b-401b-aba0-02bdbdcdaefb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:24.424202 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49167a55-c33b-401b-aba0-02bdbdcdaefb admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:24.424562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49167a55-c33b-401b-aba0-02bdbdcdaefb admin admin] PATH_INFO: `/v3/users/966e2e7af9e449dea77e6fffb77351d6` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:24.431995 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49167a55-c33b-401b-aba0-02bdbdcdaefb admin admin] RBAC: Authorizing `identity:delete_user(user_id=966e2e7af9e449dea77e6fffb77351d6)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:24.434502 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49167a55-c33b-401b-aba0-02bdbdcdaefb admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:24.496730 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-49167a55-c33b-401b-aba0-02bdbdcdaefb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '966e2e7af9e449dea77e6fffb77351d6'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:24.506295 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-49167a55-c33b-401b-aba0-02bdbdcdaefb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '966e2e7af9e449dea77e6fffb77351d6'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:24.507042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-49167a55-c33b-401b-aba0-02bdbdcdaefb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '966e2e7af9e449dea77e6fffb77351d6'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:24.523676 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-49167a55-c33b-401b-aba0-02bdbdcdaefb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '966e2e7af9e449dea77e6fffb77351d6'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:24.524475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-49167a55-c33b-401b-aba0-02bdbdcdaefb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '966e2e7af9e449dea77e6fffb77351d6'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:24.544658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1006/1963] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:24 2022] DELETE /identity/v3/users/966e2e7af9e449dea77e6fffb77351d6 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:24.930270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7127e4b-d8b3-4fd6-a689-b981fdbc893a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:24.931855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a7127e4b-d8b3-4fd6-a689-b981fdbc893a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:24.949332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7127e4b-d8b3-4fd6-a689-b981fdbc893a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:25.014740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745fa482-1cfb-4bf1-bb3c-5f5ea142919c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:25.015635 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fa482-1cfb-4bf1-bb3c-5f5ea142919c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:25.015929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fa482-1cfb-4bf1-bb3c-5f5ea142919c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:25.016214 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fa482-1cfb-4bf1-bb3c-5f5ea142919c admin admin] PATH_INFO: `/v3/projects/844ac7ef7d154666895857d591d2c08f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:25.017427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745fa482-1cfb-4bf1-bb3c-5f5ea142919c admin admin] RBAC: Authorizing `identity:delete_project(project_id=844ac7ef7d154666895857d591d2c08f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:25.019386 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745fa482-1cfb-4bf1-bb3c-5f5ea142919c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:25.077516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-745fa482-1cfb-4bf1-bb3c-5f5ea142919c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '844ac7ef7d154666895857d591d2c08f', 'request_id': 'req-745fa482-1cfb-4bf1-bb3c-5f5ea142919c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:25.080783 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-745fa482-1cfb-4bf1-bb3c-5f5ea142919c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '844ac7ef7d154666895857d591d2c08f', 'request_id': 'req-745fa482-1cfb-4bf1-bb3c-5f5ea142919c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:25.087864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-745fa482-1cfb-4bf1-bb3c-5f5ea142919c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '844ac7ef7d154666895857d591d2c08f', 'request_id': 'req-745fa482-1cfb-4bf1-bb3c-5f5ea142919c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:25.106374 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-745fa482-1cfb-4bf1-bb3c-5f5ea142919c admin admin] The token cache is being invalidate because project 844ac7ef7d154666895857d591d2c08f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:25.106759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-745fa482-1cfb-4bf1-bb3c-5f5ea142919c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:25.107390 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-745fa482-1cfb-4bf1-bb3c-5f5ea142919c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:25.109539 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 958/1964] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:24 2022] DELETE /identity/v3/projects/844ac7ef7d154666895857d591d2c08f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:25.114547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46b98f-861e-48f7-aac5-3d946ec148a2 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:25.115076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46b98f-861e-48f7-aac5-3d946ec148a2 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:25.115256 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46b98f-861e-48f7-aac5-3d946ec148a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:25.137906 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-0d46b98f-861e-48f7-aac5-3d946ec148a2 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:25.140129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0d46b98f-861e-48f7-aac5-3d946ec148a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:25.220939 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1007/1965] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:25.231310 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553805e-2b48-4617-8fd9-675f8df3a24c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:25.231867 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553805e-2b48-4617-8fd9-675f8df3a24c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:25.232512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553805e-2b48-4617-8fd9-675f8df3a24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:25.263907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f553805e-2b48-4617-8fd9-675f8df3a24c None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:25.265027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f553805e-2b48-4617-8fd9-675f8df3a24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:25.275960 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 959/1966] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:25.280408 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46b98f-861e-48f7-aac5-3d946ec148a2 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:25.291752 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:25.297224 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46b98f-861e-48f7-aac5-3d946ec148a2 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:25.305991 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc470b-ccea-4aa2-b01c-72c2749e66ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:25.307995 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc470b-ccea-4aa2-b01c-72c2749e66ee admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:25.308112 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc470b-ccea-4aa2-b01c-72c2749e66ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:25.308716 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc470b-ccea-4aa2-b01c-72c2749e66ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:25.309985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc470b-ccea-4aa2-b01c-72c2749e66ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:39:25.310695 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc470b-ccea-4aa2-b01c-72c2749e66ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:25.312587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc470b-ccea-4aa2-b01c-72c2749e66ee admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:25.338042 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553805e-2b48-4617-8fd9-675f8df3a24c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:25.342938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1008/1967] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:39:25.347764 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553805e-2b48-4617-8fd9-675f8df3a24c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:25.355384 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1fcd3a-16dc-4ab6-9065-c16c5d1be303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:25.356331 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fcd3a-16dc-4ab6-9065-c16c5d1be303 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:25.356843 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fcd3a-16dc-4ab6-9065-c16c5d1be303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:25.357335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fcd3a-16dc-4ab6-9065-c16c5d1be303 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:25.358497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1fcd3a-16dc-4ab6-9065-c16c5d1be303 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:25.361152 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1fcd3a-16dc-4ab6-9065-c16c5d1be303 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:25.396396 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 960/1968] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:25 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:39:25.402599 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc470b-ccea-4aa2-b01c-72c2749e66ee admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:25.417240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc470b-ccea-4aa2-b01c-72c2749e66ee admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:25.483570 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d42c2-bcd9-4d2d-b2fe-95666b3f3e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:25.484252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d42c2-bcd9-4d2d-b2fe-95666b3f3e58 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:25.484542 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d42c2-bcd9-4d2d-b2fe-95666b3f3e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:25.484850 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d42c2-bcd9-4d2d-b2fe-95666b3f3e58 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:25.485544 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d42c2-bcd9-4d2d-b2fe-95666b3f3e58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:25.487515 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d42c2-bcd9-4d2d-b2fe-95666b3f3e58 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:25.508029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1009/1969] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:25 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:25.512138 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1fcd3a-16dc-4ab6-9065-c16c5d1be303 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:25.522124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1fcd3a-16dc-4ab6-9065-c16c5d1be303 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:25.526715 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc2ac6-e0ba-42c9-8271-2985f5f7c4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:25.527376 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc2ac6-e0ba-42c9-8271-2985f5f7c4a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:25.527655 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc2ac6-e0ba-42c9-8271-2985f5f7c4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:25.527927 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc2ac6-e0ba-42c9-8271-2985f5f7c4a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:25.528648 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc2ac6-e0ba-42c9-8271-2985f5f7c4a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:25.530218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc2ac6-e0ba-42c9-8271-2985f5f7c4a3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:25.540945 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 961/1970] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:25.545021 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d42c2-bcd9-4d2d-b2fe-95666b3f3e58 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:25.555465 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d42c2-bcd9-4d2d-b2fe-95666b3f3e58 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:25.560815 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee37c6-b3e7-4ef8-b837-c8a452cf2751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:25.561472 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee37c6-b3e7-4ef8-b837-c8a452cf2751 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:25.561785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee37c6-b3e7-4ef8-b837-c8a452cf2751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:25.562066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee37c6-b3e7-4ef8-b837-c8a452cf2751 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:25.562837 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee37c6-b3e7-4ef8-b837-c8a452cf2751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:25.564696 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee37c6-b3e7-4ef8-b837-c8a452cf2751 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:25.576166 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1010/1971] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:25.580616 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc2ac6-e0ba-42c9-8271-2985f5f7c4a3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:25.590978 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc2ac6-e0ba-42c9-8271-2985f5f7c4a3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:25.595541 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2887fd9-a6ad-4ae2-afde-71e19412cac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:25.596420 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2887fd9-a6ad-4ae2-afde-71e19412cac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:25.596705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2887fd9-a6ad-4ae2-afde-71e19412cac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:25.596984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2887fd9-a6ad-4ae2-afde-71e19412cac8 admin admin] PATH_INFO: `/v3/projects/7856bd61b5b7433a806fa7d52249e544/users/0e7899ad3e6a403e8aca5b277068ef35/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:25.611755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2887fd9-a6ad-4ae2-afde-71e19412cac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7856bd61b5b7433a806fa7d52249e544, user_id=0e7899ad3e6a403e8aca5b277068ef35, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:25.614350 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2887fd9-a6ad-4ae2-afde-71e19412cac8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:25.627272 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 962/1972] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:25 2022] PUT /identity/v3/projects/7856bd61b5b7433a806fa7d52249e544/users/0e7899ad3e6a403e8aca5b277068ef35/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:25.638523 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af89a3-dfe7-485a-b16d-1c4cefc0a936 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:25.638847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af89a3-dfe7-485a-b16d-1c4cefc0a936 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:25.639142 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af89a3-dfe7-485a-b16d-1c4cefc0a936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:25.668105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-13af89a3-dfe7-485a-b16d-1c4cefc0a936 None None] MFA Rules not processed for user `0e7899ad3e6a403e8aca5b277068ef35`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:25.669336 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-13af89a3-dfe7-485a-b16d-1c4cefc0a936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:25.747029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1011/1973] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:25.751534 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2887fd9-a6ad-4ae2-afde-71e19412cac8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:25.770339 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:25.819832 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2887fd9-a6ad-4ae2-afde-71e19412cac8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:25.825121 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca8ff6-5e6d-4b63-82be-4b4580998a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:25.825808 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca8ff6-5e6d-4b63-82be-4b4580998a9b admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:25.826080 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca8ff6-5e6d-4b63-82be-4b4580998a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:25.826363 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca8ff6-5e6d-4b63-82be-4b4580998a9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:25.827103 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca8ff6-5e6d-4b63-82be-4b4580998a9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:25.829071 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca8ff6-5e6d-4b63-82be-4b4580998a9b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:25.860333 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 963/1974] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:25 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:25.861478 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af89a3-dfe7-485a-b16d-1c4cefc0a936 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:25.872307 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af89a3-dfe7-485a-b16d-1c4cefc0a936 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:25.936905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74731581-9f85-47e1-9068-89615801ee5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:25.937824 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74731581-9f85-47e1-9068-89615801ee5c admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:25.938175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74731581-9f85-47e1-9068-89615801ee5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:25.938542 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74731581-9f85-47e1-9068-89615801ee5c admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:25.939354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74731581-9f85-47e1-9068-89615801ee5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:25.941669 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74731581-9f85-47e1-9068-89615801ee5c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:25.949097 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:25.964867 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1012/1975] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:25 2022] POST /identity/v3/users => generated 298 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:39:25.968986 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca8ff6-5e6d-4b63-82be-4b4580998a9b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:25.982428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca8ff6-5e6d-4b63-82be-4b4580998a9b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:25.987670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21959c4d-9b53-42fa-bac6-802ba7133a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:25.988349 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21959c4d-9b53-42fa-bac6-802ba7133a54 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:25.988636 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21959c4d-9b53-42fa-bac6-802ba7133a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:25.988912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21959c4d-9b53-42fa-bac6-802ba7133a54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:25.989641 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21959c4d-9b53-42fa-bac6-802ba7133a54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:25.991282 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21959c4d-9b53-42fa-bac6-802ba7133a54 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:26.002497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 964/1976] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:26.006529 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74731581-9f85-47e1-9068-89615801ee5c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:26.017096 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74731581-9f85-47e1-9068-89615801ee5c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:26.023043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251feba-a27c-4706-bfce-f8befb1b818e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:26.023985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251feba-a27c-4706-bfce-f8befb1b818e admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:26.024216 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251feba-a27c-4706-bfce-f8befb1b818e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:26.024498 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251feba-a27c-4706-bfce-f8befb1b818e admin admin] PATH_INFO: `/v3/projects/7ca4ec716e8748a081cf1797a42fedd4/users/1feb45ddd74548b6bdef01111161c608/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:26.040533 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251feba-a27c-4706-bfce-f8befb1b818e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca4ec716e8748a081cf1797a42fedd4, user_id=1feb45ddd74548b6bdef01111161c608, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:26.043232 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251feba-a27c-4706-bfce-f8befb1b818e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:26.056752 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1013/1977] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:26 2022] PUT /identity/v3/projects/7ca4ec716e8748a081cf1797a42fedd4/users/1feb45ddd74548b6bdef01111161c608/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:26.060753 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21959c4d-9b53-42fa-bac6-802ba7133a54 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:26.073121 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21959c4d-9b53-42fa-bac6-802ba7133a54 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:26.136237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345af6a3-b54a-40ab-a08b-4c7912a5310c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:26.136938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345af6a3-b54a-40ab-a08b-4c7912a5310c admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:26.137231 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345af6a3-b54a-40ab-a08b-4c7912a5310c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:26.137505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345af6a3-b54a-40ab-a08b-4c7912a5310c admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:26.138240 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345af6a3-b54a-40ab-a08b-4c7912a5310c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:26.139885 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345af6a3-b54a-40ab-a08b-4c7912a5310c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:26.150471 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 965/1978] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:26.154570 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251feba-a27c-4706-bfce-f8befb1b818e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:26.164841 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251feba-a27c-4706-bfce-f8befb1b818e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:26.169614 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f84a33f-4bae-4f16-8e5c-ce4c18c59077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:26.170500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84a33f-4bae-4f16-8e5c-ce4c18c59077 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:26.170816 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84a33f-4bae-4f16-8e5c-ce4c18c59077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:26.171098 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84a33f-4bae-4f16-8e5c-ce4c18c59077 admin admin] PATH_INFO: `/v3/projects/7ca4ec716e8748a081cf1797a42fedd4/users/1feb45ddd74548b6bdef01111161c608/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:26.173239 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f84a33f-4bae-4f16-8e5c-ce4c18c59077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca4ec716e8748a081cf1797a42fedd4, user_id=1feb45ddd74548b6bdef01111161c608, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:26.175883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f84a33f-4bae-4f16-8e5c-ce4c18c59077 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:26.185918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1014/1979] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:26 2022] PUT /identity/v3/projects/7ca4ec716e8748a081cf1797a42fedd4/users/1feb45ddd74548b6bdef01111161c608/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:26.196237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f1897-8fa5-471b-83de-cd0ef44017f3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:26.196528 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f1897-8fa5-471b-83de-cd0ef44017f3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:26.197312 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f1897-8fa5-471b-83de-cd0ef44017f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:26.223982 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c12f1897-8fa5-471b-83de-cd0ef44017f3 None None] MFA Rules not processed for user `1feb45ddd74548b6bdef01111161c608`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:26.225178 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c12f1897-8fa5-471b-83de-cd0ef44017f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:26.292228 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 966/1980] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:26.302671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f84a33f-4bae-4f16-8e5c-ce4c18c59077 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:26.304375 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5f84a33f-4bae-4f16-8e5c-ce4c18c59077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:26.361867 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f84a33f-4bae-4f16-8e5c-ce4c18c59077 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:26.365542 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a3a07-ab22-4667-a02b-8633a70c4477 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:26.366190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a3a07-ab22-4667-a02b-8633a70c4477 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:26.366476 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a3a07-ab22-4667-a02b-8633a70c4477 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:26.366780 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a3a07-ab22-4667-a02b-8633a70c4477 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:26.425103 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a3a07-ab22-4667-a02b-8633a70c4477 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:26.426557 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a3a07-ab22-4667-a02b-8633a70c4477 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:26.436391 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1015/1981] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:26.576862 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f1897-8fa5-471b-83de-cd0ef44017f3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:26.578885 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c12f1897-8fa5-471b-83de-cd0ef44017f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:26.597030 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:26.676951 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f1897-8fa5-471b-83de-cd0ef44017f3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:26.683469 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab13f02-142f-4136-814f-b58f70762e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:26.684661 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab13f02-142f-4136-814f-b58f70762e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:26.685056 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab13f02-142f-4136-814f-b58f70762e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:26.685432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab13f02-142f-4136-814f-b58f70762e1e admin admin] PATH_INFO: `/v3/users/13f0999c6db24472ba1d83c38d43f79a` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:26.686938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab13f02-142f-4136-814f-b58f70762e1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=13f0999c6db24472ba1d83c38d43f79a)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:26.699741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab13f02-142f-4136-814f-b58f70762e1e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:26.799808 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fab13f02-142f-4136-814f-b58f70762e1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13f0999c6db24472ba1d83c38d43f79a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:26.812466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fab13f02-142f-4136-814f-b58f70762e1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13f0999c6db24472ba1d83c38d43f79a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:26.832959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fab13f02-142f-4136-814f-b58f70762e1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13f0999c6db24472ba1d83c38d43f79a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:26.849124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fab13f02-142f-4136-814f-b58f70762e1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13f0999c6db24472ba1d83c38d43f79a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:26.850316 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fab13f02-142f-4136-814f-b58f70762e1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13f0999c6db24472ba1d83c38d43f79a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:26.858983 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 967/1982] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:26 2022] DELETE /identity/v3/users/13f0999c6db24472ba1d83c38d43f79a => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:26.863498 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1699b-25f7-4b35-b7fd-fc6513a0c928 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:26.863797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1699b-25f7-4b35-b7fd-fc6513a0c928 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:26.864087 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1699b-25f7-4b35-b7fd-fc6513a0c928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:26.916511 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-86e1699b-25f7-4b35-b7fd-fc6513a0c928 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:26.917780 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-86e1699b-25f7-4b35-b7fd-fc6513a0c928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:27.090314 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1016/1983] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:27.100283 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab13f02-142f-4136-814f-b58f70762e1e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:27.101644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fab13f02-142f-4136-814f-b58f70762e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:27.158771 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1699b-25f7-4b35-b7fd-fc6513a0c928 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:27.160852 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-86e1699b-25f7-4b35-b7fd-fc6513a0c928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:27.179221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab13f02-142f-4136-814f-b58f70762e1e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:27.182412 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:27.185131 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980cf3b-a0db-4b70-816a-7eb5a3788386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:27.185863 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980cf3b-a0db-4b70-816a-7eb5a3788386 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:27.186088 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980cf3b-a0db-4b70-816a-7eb5a3788386 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:27.187138 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980cf3b-a0db-4b70-816a-7eb5a3788386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:27.202952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980cf3b-a0db-4b70-816a-7eb5a3788386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:27.204644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980cf3b-a0db-4b70-816a-7eb5a3788386 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:27.219256 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 968/1984] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:39:27.260049 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1699b-25f7-4b35-b7fd-fc6513a0c928 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:27.264548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f522428-4199-4ed9-be2f-32a19e7af076 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:27.265035 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f522428-4199-4ed9-be2f-32a19e7af076 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:27.265327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f522428-4199-4ed9-be2f-32a19e7af076 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:27.265609 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f522428-4199-4ed9-be2f-32a19e7af076 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:27.267636 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8f522428-4199-4ed9-be2f-32a19e7af076 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:27.355319 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f522428-4199-4ed9-be2f-32a19e7af076 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:27.357065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f522428-4199-4ed9-be2f-32a19e7af076 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:27.374325 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1017/1985] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:27.588710 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980cf3b-a0db-4b70-816a-7eb5a3788386 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:27.590142 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6980cf3b-a0db-4b70-816a-7eb5a3788386 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:27.610283 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980cf3b-a0db-4b70-816a-7eb5a3788386 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:27.616041 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca84e3b-f6be-4e54-9c33-c694cdb5874b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:27.616985 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca84e3b-f6be-4e54-9c33-c694cdb5874b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:27.617271 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca84e3b-f6be-4e54-9c33-c694cdb5874b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:27.617546 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca84e3b-f6be-4e54-9c33-c694cdb5874b admin admin] PATH_INFO: `/v3/projects/0c0a4de948424d6ca968d4076d700d2d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:27.618919 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca84e3b-f6be-4e54-9c33-c694cdb5874b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c0a4de948424d6ca968d4076d700d2d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:27.620915 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca84e3b-f6be-4e54-9c33-c694cdb5874b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:27.655373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:27.672987 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-aca84e3b-f6be-4e54-9c33-c694cdb5874b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c0a4de948424d6ca968d4076d700d2d', 'request_id': 'req-aca84e3b-f6be-4e54-9c33-c694cdb5874b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:27.679202 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-aca84e3b-f6be-4e54-9c33-c694cdb5874b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c0a4de948424d6ca968d4076d700d2d', 'request_id': 'req-aca84e3b-f6be-4e54-9c33-c694cdb5874b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:27.685875 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-aca84e3b-f6be-4e54-9c33-c694cdb5874b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c0a4de948424d6ca968d4076d700d2d', 'request_id': 'req-aca84e3b-f6be-4e54-9c33-c694cdb5874b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:27.698434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-aca84e3b-f6be-4e54-9c33-c694cdb5874b admin admin] The token cache is being invalidate because project 0c0a4de948424d6ca968d4076d700d2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:27.698786 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-aca84e3b-f6be-4e54-9c33-c694cdb5874b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:27.699410 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-aca84e3b-f6be-4e54-9c33-c694cdb5874b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:27.700962 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 969/1986] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:27 2022] DELETE /identity/v3/projects/0c0a4de948424d6ca968d4076d700d2d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:27.706026 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b8256-c3c5-45fe-819b-53bd98b9897d None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:27.706316 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b8256-c3c5-45fe-819b-53bd98b9897d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:27.706594 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b8256-c3c5-45fe-819b-53bd98b9897d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:27.724682 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-ef1b8256-c3c5-45fe-819b-53bd98b9897d None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:27.725949 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ef1b8256-c3c5-45fe-819b-53bd98b9897d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:27.791729 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1018/1987] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:27.799443 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8a420-9963-4e59-9a0b-76939bab82d0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:27.799811 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8a420-9963-4e59-9a0b-76939bab82d0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:27.800212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8a420-9963-4e59-9a0b-76939bab82d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:27.820225 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-fce8a420-9963-4e59-9a0b-76939bab82d0 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:27.821447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fce8a420-9963-4e59-9a0b-76939bab82d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:27.830844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 970/1988] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:27.835119 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b8256-c3c5-45fe-819b-53bd98b9897d None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:27.845619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b8256-c3c5-45fe-819b-53bd98b9897d None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:27.850802 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33bd66-a597-4406-900b-e2b6466565d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:27.851474 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33bd66-a597-4406-900b-e2b6466565d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:27.851759 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33bd66-a597-4406-900b-e2b6466565d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:27.852038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33bd66-a597-4406-900b-e2b6466565d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:27.852800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33bd66-a597-4406-900b-e2b6466565d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:39:27.853139 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33bd66-a597-4406-900b-e2b6466565d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:27.854863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33bd66-a597-4406-900b-e2b6466565d3 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:27.867334 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1019/1989] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:39:27.871950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8a420-9963-4e59-9a0b-76939bab82d0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:27.882181 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8a420-9963-4e59-9a0b-76939bab82d0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:27.887784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74466ea-a045-44e3-9b04-6090be008d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:27.888447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74466ea-a045-44e3-9b04-6090be008d3c admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:27.888914 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74466ea-a045-44e3-9b04-6090be008d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:27.889043 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74466ea-a045-44e3-9b04-6090be008d3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:27.889701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74466ea-a045-44e3-9b04-6090be008d3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:27.891680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74466ea-a045-44e3-9b04-6090be008d3c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:27.920337 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 971/1990] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:27 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:27.921435 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33bd66-a597-4406-900b-e2b6466565d3 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:27.933489 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33bd66-a597-4406-900b-e2b6466565d3 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:27.989055 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:28.017216 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cd9e1-928a-4068-a9b6-87034701e677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:28.018081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cd9e1-928a-4068-a9b6-87034701e677 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:28.018885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cd9e1-928a-4068-a9b6-87034701e677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:28.028110 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cd9e1-928a-4068-a9b6-87034701e677 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:28.029868 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015cd9e1-928a-4068-a9b6-87034701e677 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:28.032166 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015cd9e1-928a-4068-a9b6-87034701e677 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:28.067068 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1020/1991] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:27 2022] POST /identity/v3/users => generated 284 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:39:28.067178 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74466ea-a045-44e3-9b04-6090be008d3c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:28.081024 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74466ea-a045-44e3-9b04-6090be008d3c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:28.087287 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50228f89-0c73-4977-b19b-eee63d6ea90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:28.088315 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50228f89-0c73-4977-b19b-eee63d6ea90e admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:28.088612 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50228f89-0c73-4977-b19b-eee63d6ea90e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:28.088900 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50228f89-0c73-4977-b19b-eee63d6ea90e admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:28.090297 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50228f89-0c73-4977-b19b-eee63d6ea90e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:28.091980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50228f89-0c73-4977-b19b-eee63d6ea90e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:28.104445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 972/1992] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:39:28.108791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cd9e1-928a-4068-a9b6-87034701e677 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:28.120561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cd9e1-928a-4068-a9b6-87034701e677 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:28.126052 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7ac44-8afa-4739-9da2-6ad0e47746c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:28.126999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7ac44-8afa-4739-9da2-6ad0e47746c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:28.127298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7ac44-8afa-4739-9da2-6ad0e47746c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:28.127588 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7ac44-8afa-4739-9da2-6ad0e47746c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:28.128518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7ac44-8afa-4739-9da2-6ad0e47746c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:28.130401 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7ac44-8afa-4739-9da2-6ad0e47746c8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:28.146760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1021/1993] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:28.151357 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50228f89-0c73-4977-b19b-eee63d6ea90e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:28.163869 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50228f89-0c73-4977-b19b-eee63d6ea90e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:28.169576 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a892aa4-ae6d-4007-aa3c-8387d036df92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:28.170952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a892aa4-ae6d-4007-aa3c-8387d036df92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:28.171521 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a892aa4-ae6d-4007-aa3c-8387d036df92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:28.171802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a892aa4-ae6d-4007-aa3c-8387d036df92 admin admin] PATH_INFO: `/v3/projects/3c38356bdf904142aaca69087953f480/users/1146f607aae246c39d69cdeca4c9cf48/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:28.197246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a892aa4-ae6d-4007-aa3c-8387d036df92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c38356bdf904142aaca69087953f480, user_id=1146f607aae246c39d69cdeca4c9cf48, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:28.202317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a892aa4-ae6d-4007-aa3c-8387d036df92 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:28.231884 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 973/1994] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:28 2022] PUT /identity/v3/projects/3c38356bdf904142aaca69087953f480/users/1146f607aae246c39d69cdeca4c9cf48/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:28.234566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c59000-cea8-497f-8248-7c349de5c811 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:28.235047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c59000-cea8-497f-8248-7c349de5c811 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:28.235384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c59000-cea8-497f-8248-7c349de5c811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:28.268879 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-f9c59000-cea8-497f-8248-7c349de5c811 None None] MFA Rules not processed for user `1146f607aae246c39d69cdeca4c9cf48`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:28.270108 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f9c59000-cea8-497f-8248-7c349de5c811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:28.355666 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1022/1995] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:28 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:28.370782 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a892aa4-ae6d-4007-aa3c-8387d036df92 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:28.372743 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5a892aa4-ae6d-4007-aa3c-8387d036df92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:28.436566 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a892aa4-ae6d-4007-aa3c-8387d036df92 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:28.440947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a504714-600c-4e1e-a2e1-6db0f01d58ee service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8a4b0dd4e21414d91fe7bed478d15e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:28.441607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a504714-600c-4e1e-a2e1-6db0f01d58ee service glance] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:28.441896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a504714-600c-4e1e-a2e1-6db0f01d58ee service glance] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:28.442170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a504714-600c-4e1e-a2e1-6db0f01d58ee service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:28.451393 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a504714-600c-4e1e-a2e1-6db0f01d58ee service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:28.453195 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a504714-600c-4e1e-a2e1-6db0f01d58ee service glance] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:28.462788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 974/1996] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:28 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:29.719338 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c59000-cea8-497f-8248-7c349de5c811 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:29.719956 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a504714-600c-4e1e-a2e1-6db0f01d58ee service glance] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:29.720884 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f9c59000-cea8-497f-8248-7c349de5c811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:29.721372 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4a504714-600c-4e1e-a2e1-6db0f01d58ee service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:29.784396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c59000-cea8-497f-8248-7c349de5c811 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:29.786714 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:29.788243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed2e6e-84b5-4ded-b214-1189f8e34f17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:29.791109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed2e6e-84b5-4ded-b214-1189f8e34f17 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:29.791469 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed2e6e-84b5-4ded-b214-1189f8e34f17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:29.791766 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed2e6e-84b5-4ded-b214-1189f8e34f17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:29.793633 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-57ed2e6e-84b5-4ded-b214-1189f8e34f17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:29.805113 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a504714-600c-4e1e-a2e1-6db0f01d58ee service glance] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:29.812686 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba149b-1caa-49ce-90af-496977a25cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:29.813592 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba149b-1caa-49ce-90af-496977a25cad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:29.813876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba149b-1caa-49ce-90af-496977a25cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:29.814156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba149b-1caa-49ce-90af-496977a25cad admin admin] PATH_INFO: `/v3/users/68b2083a3c1343d690f632f6a1a42897` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:29.815654 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba149b-1caa-49ce-90af-496977a25cad admin admin] RBAC: Authorizing `identity:delete_user(user_id=68b2083a3c1343d690f632f6a1a42897)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:29.817766 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba149b-1caa-49ce-90af-496977a25cad admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:29.878992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed2e6e-84b5-4ded-b214-1189f8e34f17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:29.879713 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-4bba149b-1caa-49ce-90af-496977a25cad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68b2083a3c1343d690f632f6a1a42897'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:29.880455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed2e6e-84b5-4ded-b214-1189f8e34f17 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:29.889181 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-4bba149b-1caa-49ce-90af-496977a25cad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68b2083a3c1343d690f632f6a1a42897'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:29.893265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1023/1997] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:29 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:29.898760 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-4bba149b-1caa-49ce-90af-496977a25cad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68b2083a3c1343d690f632f6a1a42897'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:29.915309 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-4bba149b-1caa-49ce-90af-496977a25cad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68b2083a3c1343d690f632f6a1a42897'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:29.916354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-4bba149b-1caa-49ce-90af-496977a25cad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68b2083a3c1343d690f632f6a1a42897'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:29.923363 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 975/1998] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:29 2022] DELETE /identity/v3/users/68b2083a3c1343d690f632f6a1a42897 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:29.927883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec150d-50ef-4893-ba54-3b8bb796c655 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:29.928171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec150d-50ef-4893-ba54-3b8bb796c655 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:29.928448 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec150d-50ef-4893-ba54-3b8bb796c655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:29.942383 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:29.949443 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-03ec150d-50ef-4893-ba54-3b8bb796c655 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:29.952347 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-03ec150d-50ef-4893-ba54-3b8bb796c655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:30.042146 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1024/1999] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:39:30.051843 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba149b-1caa-49ce-90af-496977a25cad admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:30.053273 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4bba149b-1caa-49ce-90af-496977a25cad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:30.110867 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba149b-1caa-49ce-90af-496977a25cad admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:30.114594 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285bb20b-02c5-44e0-9af7-c405d33e50e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:30.115298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bb20b-02c5-44e0-9af7-c405d33e50e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:30.115594 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bb20b-02c5-44e0-9af7-c405d33e50e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:30.115883 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bb20b-02c5-44e0-9af7-c405d33e50e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:30.136407 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285bb20b-02c5-44e0-9af7-c405d33e50e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:30.137866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285bb20b-02c5-44e0-9af7-c405d33e50e0 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:30.148783 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 976/2000] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:30.268729 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec150d-50ef-4893-ba54-3b8bb796c655 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:30.270269 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-03ec150d-50ef-4893-ba54-3b8bb796c655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:30.345440 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec150d-50ef-4893-ba54-3b8bb796c655 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:30.350105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f1ae2-2f0c-4166-8bfb-ca9c5f1975d9 tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3eb7c87d5e6f4c89964a15d552010e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fa1f1afd23147e5a73e3ddb055213a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:30.351126 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f1ae2-2f0c-4166-8bfb-ca9c5f1975d9 tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:30.351277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f1ae2-2f0c-4166-8bfb-ca9c5f1975d9 tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:30.351514 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f1ae2-2f0c-4166-8bfb-ca9c5f1975d9 tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:30.353142 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f1ae2-2f0c-4166-8bfb-ca9c5f1975d9 tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:39:30.353142 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f1ae2-2f0c-4166-8bfb-ca9c5f1975d9 tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:30.354248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f1ae2-2f0c-4166-8bfb-ca9c5f1975d9 tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:30.369873 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1025/2001] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:39:30.371351 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285bb20b-02c5-44e0-9af7-c405d33e50e0 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:30.381761 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285bb20b-02c5-44e0-9af7-c405d33e50e0 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:30.387041 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f6cd2-6439-4b01-a836-f4863f48f516 tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3eb7c87d5e6f4c89964a15d552010e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fa1f1afd23147e5a73e3ddb055213a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:30.387707 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f6cd2-6439-4b01-a836-f4863f48f516 tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:30.388002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f6cd2-6439-4b01-a836-f4863f48f516 tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:30.388284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f6cd2-6439-4b01-a836-f4863f48f516 tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:30.388995 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f6cd2-6439-4b01-a836-f4863f48f516 tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:30.390926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f6cd2-6439-4b01-a836-f4863f48f516 tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:30.418610 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 977/2002] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:30 2022] POST /identity/v3/projects => generated 355 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:30.423170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f1ae2-2f0c-4166-8bfb-ca9c5f1975d9 tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:30.429441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f6cd2-6439-4b01-a836-f4863f48f516 tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:30.432176 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-170f6cd2-6439-4b01-a836-f4863f48f516 tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:30.434114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f1ae2-2f0c-4166-8bfb-ca9c5f1975d9 tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:30.495009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c214e2-a95d-4c0b-ae8a-013e9f5a00ed tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3eb7c87d5e6f4c89964a15d552010e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fa1f1afd23147e5a73e3ddb055213a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:30.495710 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c214e2-a95d-4c0b-ae8a-013e9f5a00ed tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:30.496006 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c214e2-a95d-4c0b-ae8a-013e9f5a00ed tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:30.496290 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c214e2-a95d-4c0b-ae8a-013e9f5a00ed tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:30.497054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c214e2-a95d-4c0b-ae8a-013e9f5a00ed tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:39:30.497470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c214e2-a95d-4c0b-ae8a-013e9f5a00ed tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:30.499229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c214e2-a95d-4c0b-ae8a-013e9f5a00ed tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:30.501998 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:30.508800 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f6cd2-6439-4b01-a836-f4863f48f516 tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:30.513484 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1026/2003] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:39:30.515005 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1982dd-7af8-4b61-9867-0ae34a1a1d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:30.515888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1982dd-7af8-4b61-9867-0ae34a1a1d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:30.516176 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1982dd-7af8-4b61-9867-0ae34a1a1d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:30.516458 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1982dd-7af8-4b61-9867-0ae34a1a1d9d admin admin] PATH_INFO: `/v3/projects/823dee98305b47c1b4c1f42ddb64b5ed` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:30.517673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1982dd-7af8-4b61-9867-0ae34a1a1d9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=823dee98305b47c1b4c1f42ddb64b5ed)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:30.519647 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1982dd-7af8-4b61-9867-0ae34a1a1d9d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:30.554440 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c214e2-a95d-4c0b-ae8a-013e9f5a00ed tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:30.565706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c214e2-a95d-4c0b-ae8a-013e9f5a00ed tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:30.571051 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ab1982dd-7af8-4b61-9867-0ae34a1a1d9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '823dee98305b47c1b4c1f42ddb64b5ed', 'request_id': 'req-ab1982dd-7af8-4b61-9867-0ae34a1a1d9d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:30.576713 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ab1982dd-7af8-4b61-9867-0ae34a1a1d9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '823dee98305b47c1b4c1f42ddb64b5ed', 'request_id': 'req-ab1982dd-7af8-4b61-9867-0ae34a1a1d9d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:30.582992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ab1982dd-7af8-4b61-9867-0ae34a1a1d9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '823dee98305b47c1b4c1f42ddb64b5ed', 'request_id': 'req-ab1982dd-7af8-4b61-9867-0ae34a1a1d9d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:30.611575 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ab1982dd-7af8-4b61-9867-0ae34a1a1d9d admin admin] The token cache is being invalidate because project 823dee98305b47c1b4c1f42ddb64b5ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:30.611876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ab1982dd-7af8-4b61-9867-0ae34a1a1d9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:30.612727 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ab1982dd-7af8-4b61-9867-0ae34a1a1d9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:30.613901 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:30.619238 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 978/2004] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:30 2022] DELETE /identity/v3/projects/823dee98305b47c1b4c1f42ddb64b5ed => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:30.620428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1982dd-7af8-4b61-9867-0ae34a1a1d9d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:30.622144 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ab1982dd-7af8-4b61-9867-0ae34a1a1d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:30.670149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e037c0e-1473-47c6-99af-511c7c48b78f tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3eb7c87d5e6f4c89964a15d552010e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9fa1f1afd23147e5a73e3ddb055213a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:30.671180 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e037c0e-1473-47c6-99af-511c7c48b78f tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:30.671491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e037c0e-1473-47c6-99af-511c7c48b78f tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:30.671815 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e037c0e-1473-47c6-99af-511c7c48b78f tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] PATH_INFO: `/v3/projects/e107ab48a8ab4939b14db18a17852eca` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:30.672994 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e037c0e-1473-47c6-99af-511c7c48b78f tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e107ab48a8ab4939b14db18a17852eca)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:30.675232 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e037c0e-1473-47c6-99af-511c7c48b78f tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:30.707119 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1982dd-7af8-4b61-9867-0ae34a1a1d9d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:30.712878 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f8306-c39c-4d84-8c43-0e2441fd852c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:30.715154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f8306-c39c-4d84-8c43-0e2441fd852c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:30.715515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f8306-c39c-4d84-8c43-0e2441fd852c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:30.715871 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f8306-c39c-4d84-8c43-0e2441fd852c admin admin] PATH_INFO: `/v3/users/d8c0f3fbf1e84c1b8a7fc8fb5225c8b4` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:30.717117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f8306-c39c-4d84-8c43-0e2441fd852c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8c0f3fbf1e84c1b8a7fc8fb5225c8b4)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:30.719106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f8306-c39c-4d84-8c43-0e2441fd852c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:30.729751 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3e037c0e-1473-47c6-99af-511c7c48b78f tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e107ab48a8ab4939b14db18a17852eca', 'request_id': 'req-3e037c0e-1473-47c6-99af-511c7c48b78f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:30.735842 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3e037c0e-1473-47c6-99af-511c7c48b78f tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e107ab48a8ab4939b14db18a17852eca', 'request_id': 'req-3e037c0e-1473-47c6-99af-511c7c48b78f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:30.752794 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3e037c0e-1473-47c6-99af-511c7c48b78f tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e107ab48a8ab4939b14db18a17852eca', 'request_id': 'req-3e037c0e-1473-47c6-99af-511c7c48b78f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:30.765444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3e037c0e-1473-47c6-99af-511c7c48b78f tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] The token cache is being invalidate because project e107ab48a8ab4939b14db18a17852eca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:30.766060 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3e037c0e-1473-47c6-99af-511c7c48b78f tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:30.766839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3e037c0e-1473-47c6-99af-511c7c48b78f tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:30.768716 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1027/2005] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:30 2022] DELETE /identity/v3/projects/e107ab48a8ab4939b14db18a17852eca => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:30.776319 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e90f8306-c39c-4d84-8c43-0e2441fd852c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8c0f3fbf1e84c1b8a7fc8fb5225c8b4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:30.784904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e90f8306-c39c-4d84-8c43-0e2441fd852c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8c0f3fbf1e84c1b8a7fc8fb5225c8b4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:30.793228 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e90f8306-c39c-4d84-8c43-0e2441fd852c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8c0f3fbf1e84c1b8a7fc8fb5225c8b4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:30.809073 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e90f8306-c39c-4d84-8c43-0e2441fd852c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8c0f3fbf1e84c1b8a7fc8fb5225c8b4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:30.809717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e90f8306-c39c-4d84-8c43-0e2441fd852c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8c0f3fbf1e84c1b8a7fc8fb5225c8b4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:30.815913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 979/2006] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:30 2022] DELETE /identity/v3/users/d8c0f3fbf1e84c1b8a7fc8fb5225c8b4 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:30.985867 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e037c0e-1473-47c6-99af-511c7c48b78f tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:30.987710 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3e037c0e-1473-47c6-99af-511c7c48b78f tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:31.090965 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e037c0e-1473-47c6-99af-511c7c48b78f tempest-VolumeQuotaClassesTest-1309794830 tempest-VolumeQuotaClassesTest-1309794830-project-admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:31.180775 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8be4d-834f-4d96-be74-f27f69f0f4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:31.181871 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8be4d-834f-4d96-be74-f27f69f0f4a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:31.182357 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8be4d-834f-4d96-be74-f27f69f0f4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:31.182680 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8be4d-834f-4d96-be74-f27f69f0f4a6 admin admin] PATH_INFO: `/v3/users/1307f4f9c84f4568a84f10142df9f3d8` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:31.184063 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8be4d-834f-4d96-be74-f27f69f0f4a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1307f4f9c84f4568a84f10142df9f3d8)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:31.186192 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8be4d-834f-4d96-be74-f27f69f0f4a6 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:31.198928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f8306-c39c-4d84-8c43-0e2441fd852c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:31.200356 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e90f8306-c39c-4d84-8c43-0e2441fd852c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:31.207979 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:31.210175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:31.219717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f8306-c39c-4d84-8c43-0e2441fd852c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:31.228713 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d3fa7-ddf4-43fd-a47a-83e0417195d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:31.230046 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d3fa7-ddf4-43fd-a47a-83e0417195d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:31.230342 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d3fa7-ddf4-43fd-a47a-83e0417195d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:31.230635 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d3fa7-ddf4-43fd-a47a-83e0417195d4 admin admin] PATH_INFO: `/v3/projects/ef0be99ab21c43219b09225daefd7300` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:31.232253 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d3fa7-ddf4-43fd-a47a-83e0417195d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef0be99ab21c43219b09225daefd7300)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:31.234848 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d3fa7-ddf4-43fd-a47a-83e0417195d4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:31.275824 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cdc8be4d-834f-4d96-be74-f27f69f0f4a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1307f4f9c84f4568a84f10142df9f3d8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:31.284390 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cdc8be4d-834f-4d96-be74-f27f69f0f4a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1307f4f9c84f4568a84f10142df9f3d8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:31.285770 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cdc8be4d-834f-4d96-be74-f27f69f0f4a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1307f4f9c84f4568a84f10142df9f3d8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:31.312426 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cdc8be4d-834f-4d96-be74-f27f69f0f4a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1307f4f9c84f4568a84f10142df9f3d8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:31.313224 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-cdc8be4d-834f-4d96-be74-f27f69f0f4a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1307f4f9c84f4568a84f10142df9f3d8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:31.319160 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ff2d3fa7-ddf4-43fd-a47a-83e0417195d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef0be99ab21c43219b09225daefd7300', 'request_id': 'req-ff2d3fa7-ddf4-43fd-a47a-83e0417195d4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:31.325141 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ff2d3fa7-ddf4-43fd-a47a-83e0417195d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef0be99ab21c43219b09225daefd7300', 'request_id': 'req-ff2d3fa7-ddf4-43fd-a47a-83e0417195d4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:31.326916 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1028/2007] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:30 2022] DELETE /identity/v3/users/1307f4f9c84f4568a84f10142df9f3d8 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:31.331223 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959dfb5-60d2-4472-a2d2-d2b46ebef00a None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:31.331522 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959dfb5-60d2-4472-a2d2-d2b46ebef00a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:31.331806 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959dfb5-60d2-4472-a2d2-d2b46ebef00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:31.331941 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ff2d3fa7-ddf4-43fd-a47a-83e0417195d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef0be99ab21c43219b09225daefd7300', 'request_id': 'req-ff2d3fa7-ddf4-43fd-a47a-83e0417195d4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:31.345308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ff2d3fa7-ddf4-43fd-a47a-83e0417195d4 admin admin] The token cache is being invalidate because project ef0be99ab21c43219b09225daefd7300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:31.345607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ff2d3fa7-ddf4-43fd-a47a-83e0417195d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:31.346293 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ff2d3fa7-ddf4-43fd-a47a-83e0417195d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:31.347993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 980/2008] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:31 2022] DELETE /identity/v3/projects/ef0be99ab21c43219b09225daefd7300 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:31.350957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-7959dfb5-60d2-4472-a2d2-d2b46ebef00a None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:31.351753 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d3fa7-ddf4-43fd-a47a-83e0417195d4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:31.352182 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7959dfb5-60d2-4472-a2d2-d2b46ebef00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:31.353249 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ff2d3fa7-ddf4-43fd-a47a-83e0417195d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:31.414882 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1029/2009] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:39:31.419831 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d3fa7-ddf4-43fd-a47a-83e0417195d4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:31.424485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959dfb5-60d2-4472-a2d2-d2b46ebef00a None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:31.424790 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0dab4-d78e-4273-960f-7755cd78c7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:31.425682 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0dab4-d78e-4273-960f-7755cd78c7af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:31.425860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7959dfb5-60d2-4472-a2d2-d2b46ebef00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:31.425959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0dab4-d78e-4273-960f-7755cd78c7af admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:31.426235 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0dab4-d78e-4273-960f-7755cd78c7af admin admin] PATH_INFO: `/v3/users/99c9b3aa5fcc484dbfa0e838c84cc7e4` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:31.427492 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0dab4-d78e-4273-960f-7755cd78c7af admin admin] RBAC: Authorizing `identity:delete_user(user_id=99c9b3aa5fcc484dbfa0e838c84cc7e4)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:31.429418 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0dab4-d78e-4273-960f-7755cd78c7af admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:31.481803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fee0dab4-d78e-4273-960f-7755cd78c7af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99c9b3aa5fcc484dbfa0e838c84cc7e4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:31.489530 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fee0dab4-d78e-4273-960f-7755cd78c7af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99c9b3aa5fcc484dbfa0e838c84cc7e4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:31.496904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fee0dab4-d78e-4273-960f-7755cd78c7af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99c9b3aa5fcc484dbfa0e838c84cc7e4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:31.498488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959dfb5-60d2-4472-a2d2-d2b46ebef00a None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:31.510298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78969161-0e2a-4428-ad7c-5734f23eb8af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:31.510623 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fee0dab4-d78e-4273-960f-7755cd78c7af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99c9b3aa5fcc484dbfa0e838c84cc7e4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:31.511003 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78969161-0e2a-4428-ad7c-5734f23eb8af service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:31.511295 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78969161-0e2a-4428-ad7c-5734f23eb8af service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:31.511579 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78969161-0e2a-4428-ad7c-5734f23eb8af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:31.511747 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fee0dab4-d78e-4273-960f-7755cd78c7af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99c9b3aa5fcc484dbfa0e838c84cc7e4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:31.521940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78969161-0e2a-4428-ad7c-5734f23eb8af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:31.522280 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 981/2010] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:31 2022] DELETE /identity/v3/users/99c9b3aa5fcc484dbfa0e838c84cc7e4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:31.523953 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78969161-0e2a-4428-ad7c-5734f23eb8af service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:31.537226 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1030/2011] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:39:31.981375 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0dab4-d78e-4273-960f-7755cd78c7af admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:31.982937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fee0dab4-d78e-4273-960f-7755cd78c7af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:31.994497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:32.000467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78969161-0e2a-4428-ad7c-5734f23eb8af service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:32.002754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-78969161-0e2a-4428-ad7c-5734f23eb8af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:32.017637 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78969161-0e2a-4428-ad7c-5734f23eb8af service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:32.058683 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:32.085388 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb0d3f-7f17-40ce-836f-387a8ebd8e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:32.086450 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb0d3f-7f17-40ce-836f-387a8ebd8e74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:32.086876 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb0d3f-7f17-40ce-836f-387a8ebd8e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:32.087252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb0d3f-7f17-40ce-836f-387a8ebd8e74 admin admin] PATH_INFO: `/v3/projects/135d9c578ad04be69c691d4b43ad507c` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:32.088500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb0d3f-7f17-40ce-836f-387a8ebd8e74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=135d9c578ad04be69c691d4b43ad507c)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:32.090584 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb0d3f-7f17-40ce-836f-387a8ebd8e74 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:32.091437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0dab4-d78e-4273-960f-7755cd78c7af admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:32.096289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2431cd1-eca0-4ab3-8927-f04e4347be75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:32.097341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2431cd1-eca0-4ab3-8927-f04e4347be75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:32.097643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2431cd1-eca0-4ab3-8927-f04e4347be75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:32.097947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2431cd1-eca0-4ab3-8927-f04e4347be75 admin admin] PATH_INFO: `/v3/projects/7a7b16d2fe324bf3ae9afbd120e3e4e3` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:32.099160 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2431cd1-eca0-4ab3-8927-f04e4347be75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a7b16d2fe324bf3ae9afbd120e3e4e3)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:32.101201 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2431cd1-eca0-4ab3-8927-f04e4347be75 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:32.144444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c7bb0d3f-7f17-40ce-836f-387a8ebd8e74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '135d9c578ad04be69c691d4b43ad507c', 'request_id': 'req-c7bb0d3f-7f17-40ce-836f-387a8ebd8e74'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:32.151702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c7bb0d3f-7f17-40ce-836f-387a8ebd8e74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '135d9c578ad04be69c691d4b43ad507c', 'request_id': 'req-c7bb0d3f-7f17-40ce-836f-387a8ebd8e74'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:32.158591 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c7bb0d3f-7f17-40ce-836f-387a8ebd8e74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '135d9c578ad04be69c691d4b43ad507c', 'request_id': 'req-c7bb0d3f-7f17-40ce-836f-387a8ebd8e74'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:32.164884 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e2431cd1-eca0-4ab3-8927-f04e4347be75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a7b16d2fe324bf3ae9afbd120e3e4e3', 'request_id': 'req-e2431cd1-eca0-4ab3-8927-f04e4347be75'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:32.171599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e2431cd1-eca0-4ab3-8927-f04e4347be75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a7b16d2fe324bf3ae9afbd120e3e4e3', 'request_id': 'req-e2431cd1-eca0-4ab3-8927-f04e4347be75'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:32.172502 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c7bb0d3f-7f17-40ce-836f-387a8ebd8e74 admin admin] The token cache is being invalidate because project 135d9c578ad04be69c691d4b43ad507c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:32.172787 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c7bb0d3f-7f17-40ce-836f-387a8ebd8e74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:32.173428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c7bb0d3f-7f17-40ce-836f-387a8ebd8e74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:32.175079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1031/2012] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:31 2022] DELETE /identity/v3/projects/135d9c578ad04be69c691d4b43ad507c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:32.179478 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb0d3f-7f17-40ce-836f-387a8ebd8e74 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:32.181520 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c7bb0d3f-7f17-40ce-836f-387a8ebd8e74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:32.182450 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e2431cd1-eca0-4ab3-8927-f04e4347be75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a7b16d2fe324bf3ae9afbd120e3e4e3', 'request_id': 'req-e2431cd1-eca0-4ab3-8927-f04e4347be75'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:32.196931 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e2431cd1-eca0-4ab3-8927-f04e4347be75 admin admin] The token cache is being invalidate because project 7a7b16d2fe324bf3ae9afbd120e3e4e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:32.197222 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e2431cd1-eca0-4ab3-8927-f04e4347be75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:32.197852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e2431cd1-eca0-4ab3-8927-f04e4347be75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:32.199444 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 982/2013] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:31 2022] DELETE /identity/v3/projects/7a7b16d2fe324bf3ae9afbd120e3e4e3 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:32.205430 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c40b3-96ec-4229-b656-5378f5ed1872 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:32.205734 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c40b3-96ec-4229-b656-5378f5ed1872 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:32.206022 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c40b3-96ec-4229-b656-5378f5ed1872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:32.224939 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-229c40b3-96ec-4229-b656-5378f5ed1872 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:32.226278 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-229c40b3-96ec-4229-b656-5378f5ed1872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:32.250706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb0d3f-7f17-40ce-836f-387a8ebd8e74 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:32.288117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 983/2014] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:32.296958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d32fd8-38c6-4c5a-95b6-8414828a6943 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:32.297254 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d32fd8-38c6-4c5a-95b6-8414828a6943 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:32.297535 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d32fd8-38c6-4c5a-95b6-8414828a6943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:32.307609 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80efb9f6-368d-4631-a99b-d86cbda57112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:32.308493 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efb9f6-368d-4631-a99b-d86cbda57112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:32.308782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efb9f6-368d-4631-a99b-d86cbda57112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:32.309065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efb9f6-368d-4631-a99b-d86cbda57112 admin admin] PATH_INFO: `/v3/users/3eb7c87d5e6f4c89964a15d552010e73` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:32.310296 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80efb9f6-368d-4631-a99b-d86cbda57112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3eb7c87d5e6f4c89964a15d552010e73)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:32.312421 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80efb9f6-368d-4631-a99b-d86cbda57112 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:32.317166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-16d32fd8-38c6-4c5a-95b6-8414828a6943 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:32.318520 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-16d32fd8-38c6-4c5a-95b6-8414828a6943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:32.328677 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:32.329543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 984/2015] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:39:32.333608 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d32fd8-38c6-4c5a-95b6-8414828a6943 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:32.348741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d32fd8-38c6-4c5a-95b6-8414828a6943 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:32.355442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cda349-eaec-4dbc-8b0a-4e9f2279164a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:32.356196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cda349-eaec-4dbc-8b0a-4e9f2279164a admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:32.356489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cda349-eaec-4dbc-8b0a-4e9f2279164a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:32.356824 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cda349-eaec-4dbc-8b0a-4e9f2279164a admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:32.357596 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cda349-eaec-4dbc-8b0a-4e9f2279164a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:39:32.357912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cda349-eaec-4dbc-8b0a-4e9f2279164a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:32.359742 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cda349-eaec-4dbc-8b0a-4e9f2279164a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:32.376410 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 985/2016] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:39:32.378864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cda349-eaec-4dbc-8b0a-4e9f2279164a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:32.387236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-80efb9f6-368d-4631-a99b-d86cbda57112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3eb7c87d5e6f4c89964a15d552010e73'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:32.395605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cda349-eaec-4dbc-8b0a-4e9f2279164a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:32.397407 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-80efb9f6-368d-4631-a99b-d86cbda57112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3eb7c87d5e6f4c89964a15d552010e73'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:32.398320 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-80efb9f6-368d-4631-a99b-d86cbda57112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3eb7c87d5e6f4c89964a15d552010e73'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:32.404510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665d874-658e-4607-8856-811b9b44fb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:32.405641 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665d874-658e-4607-8856-811b9b44fb63 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:32.405928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665d874-658e-4607-8856-811b9b44fb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:32.406211 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665d874-658e-4607-8856-811b9b44fb63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:32.406949 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665d874-658e-4607-8856-811b9b44fb63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:32.409408 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665d874-658e-4607-8856-811b9b44fb63 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:32.420249 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-80efb9f6-368d-4631-a99b-d86cbda57112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3eb7c87d5e6f4c89964a15d552010e73'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:32.421519 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-80efb9f6-368d-4631-a99b-d86cbda57112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3eb7c87d5e6f4c89964a15d552010e73'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:32.425297 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:32.437997 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1032/2017] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:32 2022] DELETE /identity/v3/users/3eb7c87d5e6f4c89964a15d552010e73 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:32.451975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 986/2018] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:32 2022] POST /identity/v3/projects => generated 391 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:32.452838 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80efb9f6-368d-4631-a99b-d86cbda57112 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:32.454731 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-80efb9f6-368d-4631-a99b-d86cbda57112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:32.536312 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80efb9f6-368d-4631-a99b-d86cbda57112 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:32.623736 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448a732-27a5-4f04-a5de-d103a8b8e806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:32.624654 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448a732-27a5-4f04-a5de-d103a8b8e806 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:32.625503 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448a732-27a5-4f04-a5de-d103a8b8e806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:32.625895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448a732-27a5-4f04-a5de-d103a8b8e806 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:32.627030 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7448a732-27a5-4f04-a5de-d103a8b8e806 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:32.629559 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7448a732-27a5-4f04-a5de-d103a8b8e806 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:32.657403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1033/2019] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:32 2022] POST /identity/v3/users => generated 296 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:32.661247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665d874-658e-4607-8856-811b9b44fb63 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:32.691821 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665d874-658e-4607-8856-811b9b44fb63 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:32.731927 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb53d93-7a48-4391-b82b-8971ebee29b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:32.733009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb53d93-7a48-4391-b82b-8971ebee29b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:32.733525 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb53d93-7a48-4391-b82b-8971ebee29b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:32.734027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb53d93-7a48-4391-b82b-8971ebee29b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:32.735936 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb53d93-7a48-4391-b82b-8971ebee29b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:32.738507 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb53d93-7a48-4391-b82b-8971ebee29b9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:32.783558 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 987/2020] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:32.786428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448a732-27a5-4f04-a5de-d103a8b8e806 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:32.802165 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448a732-27a5-4f04-a5de-d103a8b8e806 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:32.813060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb53d93-7a48-4391-b82b-8971ebee29b9 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:32.816325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5bb53d93-7a48-4391-b82b-8971ebee29b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:32.820389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010a8b8-14a8-4fa4-83d0-9f31c093603f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:32.821572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010a8b8-14a8-4fa4-83d0-9f31c093603f admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:32.822355 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010a8b8-14a8-4fa4-83d0-9f31c093603f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:32.822666 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010a8b8-14a8-4fa4-83d0-9f31c093603f admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:32.823443 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010a8b8-14a8-4fa4-83d0-9f31c093603f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:32.825432 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010a8b8-14a8-4fa4-83d0-9f31c093603f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:32.843136 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb53d93-7a48-4391-b82b-8971ebee29b9 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:32.853748 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1034/2021] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:32.854659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010a8b8-14a8-4fa4-83d0-9f31c093603f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:32.872771 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384e305-e18e-4d91-bb6d-09f0ab78206a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:32.877819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384e305-e18e-4d91-bb6d-09f0ab78206a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:32.878315 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384e305-e18e-4d91-bb6d-09f0ab78206a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:32.879803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384e305-e18e-4d91-bb6d-09f0ab78206a admin admin] PATH_INFO: `/v3/projects/9fa1f1afd23147e5a73e3ddb055213a8` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:32.884207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384e305-e18e-4d91-bb6d-09f0ab78206a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fa1f1afd23147e5a73e3ddb055213a8)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:32.889366 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384e305-e18e-4d91-bb6d-09f0ab78206a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:32.891191 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010a8b8-14a8-4fa4-83d0-9f31c093603f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:32.907317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11ffb6-66d2-445f-b5b7-23c00cc66f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:32.909447 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11ffb6-66d2-445f-b5b7-23c00cc66f60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:32.909740 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11ffb6-66d2-445f-b5b7-23c00cc66f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:32.910017 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11ffb6-66d2-445f-b5b7-23c00cc66f60 admin admin] PATH_INFO: `/v3/projects/482830802c2444638f044e2657862ddb/users/fb5e15195e724bffb84f70bf8d822f7a/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:32.957050 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11ffb6-66d2-445f-b5b7-23c00cc66f60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=482830802c2444638f044e2657862ddb, user_id=fb5e15195e724bffb84f70bf8d822f7a, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:32.960765 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11ffb6-66d2-445f-b5b7-23c00cc66f60 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:32.964973 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:32.988456 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7384e305-e18e-4d91-bb6d-09f0ab78206a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fa1f1afd23147e5a73e3ddb055213a8', 'request_id': 'req-7384e305-e18e-4d91-bb6d-09f0ab78206a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:32.991287 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7384e305-e18e-4d91-bb6d-09f0ab78206a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fa1f1afd23147e5a73e3ddb055213a8', 'request_id': 'req-7384e305-e18e-4d91-bb6d-09f0ab78206a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:32.998694 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1035/2022] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:32 2022] PUT /identity/v3/projects/482830802c2444638f044e2657862ddb/users/fb5e15195e724bffb84f70bf8d822f7a/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:32.999925 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7384e305-e18e-4d91-bb6d-09f0ab78206a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fa1f1afd23147e5a73e3ddb055213a8', 'request_id': 'req-7384e305-e18e-4d91-bb6d-09f0ab78206a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:33.011718 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab3763-e1c6-4b74-bc71-dfe7c5e6c657 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:33.012420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab3763-e1c6-4b74-bc71-dfe7c5e6c657 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:33.013702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab3763-e1c6-4b74-bc71-dfe7c5e6c657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:33.039355 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7384e305-e18e-4d91-bb6d-09f0ab78206a admin admin] The token cache is being invalidate because project 9fa1f1afd23147e5a73e3ddb055213a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:33.039657 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7384e305-e18e-4d91-bb6d-09f0ab78206a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:33.040921 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7384e305-e18e-4d91-bb6d-09f0ab78206a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:33.045703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-1cab3763-e1c6-4b74-bc71-dfe7c5e6c657 None None] MFA Rules not processed for user `fb5e15195e724bffb84f70bf8d822f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:33.046214 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 988/2023] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:32 2022] DELETE /identity/v3/projects/9fa1f1afd23147e5a73e3ddb055213a8 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:33.047442 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1cab3763-e1c6-4b74-bc71-dfe7c5e6c657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:33.052640 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180f5c0-6186-4cff-93e1-b0c6b36242a6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:33.052835 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180f5c0-6186-4cff-93e1-b0c6b36242a6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:33.053115 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180f5c0-6186-4cff-93e1-b0c6b36242a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:33.101047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7180f5c0-6186-4cff-93e1-b0c6b36242a6 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:33.102981 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7180f5c0-6186-4cff-93e1-b0c6b36242a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:33.170683 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1036/2024] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:33.171839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab3763-e1c6-4b74-bc71-dfe7c5e6c657 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:33.173702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1cab3763-e1c6-4b74-bc71-dfe7c5e6c657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:33.263484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 989/2025] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:33.271860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e7ba2-7838-4920-9881-ed48a3dc7447 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:33.272154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e7ba2-7838-4920-9881-ed48a3dc7447 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:33.272437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e7ba2-7838-4920-9881-ed48a3dc7447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:33.308976 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a63e7ba2-7838-4920-9881-ed48a3dc7447 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:33.310253 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a63e7ba2-7838-4920-9881-ed48a3dc7447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:33.336514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 990/2026] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:39:33.337426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e7ba2-7838-4920-9881-ed48a3dc7447 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:33.348873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e7ba2-7838-4920-9881-ed48a3dc7447 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:33.351215 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab3763-e1c6-4b74-bc71-dfe7c5e6c657 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:33.353966 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87538a-d2e0-45e7-a4ed-072f0a4a31bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:33.354831 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87538a-d2e0-45e7-a4ed-072f0a4a31bb admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:33.355276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87538a-d2e0-45e7-a4ed-072f0a4a31bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:33.355568 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87538a-d2e0-45e7-a4ed-072f0a4a31bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:33.356311 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b87538a-d2e0-45e7-a4ed-072f0a4a31bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:39:33.356681 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b87538a-d2e0-45e7-a4ed-072f0a4a31bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:33.358071 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d47701-49dd-4c02-83cd-e7be84623769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:33.358327 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b87538a-d2e0-45e7-a4ed-072f0a4a31bb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:33.359002 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d47701-49dd-4c02-83cd-e7be84623769 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:33.359360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d47701-49dd-4c02-83cd-e7be84623769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:33.359710 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d47701-49dd-4c02-83cd-e7be84623769 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:33.360848 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d47701-49dd-4c02-83cd-e7be84623769 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:33.364193 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d47701-49dd-4c02-83cd-e7be84623769 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:33.372070 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 991/2027] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:39:33.377460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87538a-d2e0-45e7-a4ed-072f0a4a31bb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:33.389100 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87538a-d2e0-45e7-a4ed-072f0a4a31bb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:33.412035 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412ff7f-8653-4641-a26f-c00895743c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:33.412731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412ff7f-8653-4641-a26f-c00895743c8d admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:33.413024 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412ff7f-8653-4641-a26f-c00895743c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:33.413337 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412ff7f-8653-4641-a26f-c00895743c8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:33.414450 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412ff7f-8653-4641-a26f-c00895743c8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:33.416488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412ff7f-8653-4641-a26f-c00895743c8d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:33.428698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1037/2028] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:33 2022] POST /identity/v3/projects => generated 393 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:33.433367 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d47701-49dd-4c02-83cd-e7be84623769 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:33.461608 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 992/2029] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:33 2022] POST /identity/v3/projects => generated 401 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:33.464518 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412ff7f-8653-4641-a26f-c00895743c8d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:33.515198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:33.544662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d47701-49dd-4c02-83cd-e7be84623769 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:33.566466 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:33.581986 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412ff7f-8653-4641-a26f-c00895743c8d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:33.588374 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865e630-50fa-4a63-8d62-fa4817141548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:33.589367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865e630-50fa-4a63-8d62-fa4817141548 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:33.589660 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865e630-50fa-4a63-8d62-fa4817141548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:33.589936 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865e630-50fa-4a63-8d62-fa4817141548 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:33.590666 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5865e630-50fa-4a63-8d62-fa4817141548 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:33.592618 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5865e630-50fa-4a63-8d62-fa4817141548 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:33.624295 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 993/2030] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:33 2022] POST /identity/v3/users => generated 301 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:33.629118 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865e630-50fa-4a63-8d62-fa4817141548 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:33.641695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865e630-50fa-4a63-8d62-fa4817141548 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:33.642368 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f1bb2-85b7-433d-a64c-461a4f26a62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:33.643086 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f1bb2-85b7-433d-a64c-461a4f26a62c admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:33.643368 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f1bb2-85b7-433d-a64c-461a4f26a62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:33.643643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f1bb2-85b7-433d-a64c-461a4f26a62c admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:33.644340 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f1bb2-85b7-433d-a64c-461a4f26a62c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:33.646263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f1bb2-85b7-433d-a64c-461a4f26a62c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:33.647048 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5b97f-90c2-42dc-8141-0ecb49a134b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:33.647992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5b97f-90c2-42dc-8141-0ecb49a134b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:33.648636 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5b97f-90c2-42dc-8141-0ecb49a134b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:33.649197 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5b97f-90c2-42dc-8141-0ecb49a134b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:33.650206 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5b97f-90c2-42dc-8141-0ecb49a134b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:33.652398 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5b97f-90c2-42dc-8141-0ecb49a134b7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:33.670994 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 994/2031] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:33.675015 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5b97f-90c2-42dc-8141-0ecb49a134b7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:33.683985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1038/2032] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:33 2022] POST /identity/v3/users => generated 303 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:33.686850 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5b97f-90c2-42dc-8141-0ecb49a134b7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:33.689540 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f1bb2-85b7-433d-a64c-461a4f26a62c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:33.698973 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe62189-25c0-4307-ab27-49473e044300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:33.700404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe62189-25c0-4307-ab27-49473e044300 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:33.700866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe62189-25c0-4307-ab27-49473e044300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:33.701389 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe62189-25c0-4307-ab27-49473e044300 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:33.702317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe62189-25c0-4307-ab27-49473e044300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:33.706151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe62189-25c0-4307-ab27-49473e044300 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:33.721153 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f1bb2-85b7-433d-a64c-461a4f26a62c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:33.725186 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 995/2033] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:39:33.729698 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe62189-25c0-4307-ab27-49473e044300 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:33.732521 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0d568-cd59-4db1-92ac-731578df969e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:33.733488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0d568-cd59-4db1-92ac-731578df969e admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:33.734246 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0d568-cd59-4db1-92ac-731578df969e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:33.734246 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0d568-cd59-4db1-92ac-731578df969e admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:33.735502 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac0d568-cd59-4db1-92ac-731578df969e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:33.737969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac0d568-cd59-4db1-92ac-731578df969e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:33.744015 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe62189-25c0-4307-ab27-49473e044300 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:33.749692 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1cfc2-5073-40bd-85c0-b1888c017559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:33.750993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1cfc2-5073-40bd-85c0-b1888c017559 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:33.751556 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1cfc2-5073-40bd-85c0-b1888c017559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:33.752226 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1cfc2-5073-40bd-85c0-b1888c017559 admin admin] PATH_INFO: `/v3/projects/cf833e5e5896499391c08df9c0dbf6dd/users/90f539bd531548f09c8c40e909ca356e/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:33.752997 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1039/2034] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:39:33.758256 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0d568-cd59-4db1-92ac-731578df969e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:33.771933 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0d568-cd59-4db1-92ac-731578df969e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:33.775383 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e1cfc2-5073-40bd-85c0-b1888c017559 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf833e5e5896499391c08df9c0dbf6dd, user_id=90f539bd531548f09c8c40e909ca356e, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:33.781166 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446373f-bef2-4b5f-beb2-b58ef4aaba2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:33.781327 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e1cfc2-5073-40bd-85c0-b1888c017559 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:33.782608 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446373f-bef2-4b5f-beb2-b58ef4aaba2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:33.783302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446373f-bef2-4b5f-beb2-b58ef4aaba2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:33.783777 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446373f-bef2-4b5f-beb2-b58ef4aaba2b admin admin] PATH_INFO: `/v3/projects/5df8290f2eaa4c0dac5d354d0358aeac/users/e25194110c9f4b649ab1894676d025e4/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:33.799716 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 996/2035] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:33 2022] PUT /identity/v3/projects/cf833e5e5896499391c08df9c0dbf6dd/users/90f539bd531548f09c8c40e909ca356e/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:33.805675 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446373f-bef2-4b5f-beb2-b58ef4aaba2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df8290f2eaa4c0dac5d354d0358aeac, user_id=e25194110c9f4b649ab1894676d025e4, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:33.810630 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446373f-bef2-4b5f-beb2-b58ef4aaba2b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:33.812036 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db7749-f095-46b0-9762-112ad64a4a7d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:33.812340 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db7749-f095-46b0-9762-112ad64a4a7d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:33.812643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db7749-f095-46b0-9762-112ad64a4a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:33.834451 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1040/2036] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:33 2022] PUT /identity/v3/projects/5df8290f2eaa4c0dac5d354d0358aeac/users/e25194110c9f4b649ab1894676d025e4/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:33.838920 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446373f-bef2-4b5f-beb2-b58ef4aaba2b admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:33.871407 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-44db7749-f095-46b0-9762-112ad64a4a7d None None] MFA Rules not processed for user `90f539bd531548f09c8c40e909ca356e`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:33.873598 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-44db7749-f095-46b0-9762-112ad64a4a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:33.972200 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446373f-bef2-4b5f-beb2-b58ef4aaba2b admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:33.983314 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193c649-a5d9-4fba-ac7e-48874995ebd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:33.984300 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193c649-a5d9-4fba-ac7e-48874995ebd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:33.984626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193c649-a5d9-4fba-ac7e-48874995ebd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:33.984987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193c649-a5d9-4fba-ac7e-48874995ebd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:33.985750 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 997/2037] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:33 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:33.986888 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193c649-a5d9-4fba-ac7e-48874995ebd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:33.988762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193c649-a5d9-4fba-ac7e-48874995ebd6 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:33.991761 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44db7749-f095-46b0-9762-112ad64a4a7d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:34.001450 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1041/2038] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:34.005244 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44db7749-f095-46b0-9762-112ad64a4a7d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:34.005828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193c649-a5d9-4fba-ac7e-48874995ebd6 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:34.011165 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c17f6-16a3-4eec-866e-41f68e53bf5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:34.011839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c17f6-16a3-4eec-866e-41f68e53bf5b admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:34.012118 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c17f6-16a3-4eec-866e-41f68e53bf5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:34.012553 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c17f6-16a3-4eec-866e-41f68e53bf5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:34.013279 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c17f6-16a3-4eec-866e-41f68e53bf5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:34.015298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c17f6-16a3-4eec-866e-41f68e53bf5b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:34.017999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193c649-a5d9-4fba-ac7e-48874995ebd6 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:34.027333 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad269a4-4b99-4ead-b6cb-d0c751f4d2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:34.028496 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad269a4-4b99-4ead-b6cb-d0c751f4d2ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:34.028957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad269a4-4b99-4ead-b6cb-d0c751f4d2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:34.029782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad269a4-4b99-4ead-b6cb-d0c751f4d2ff admin admin] PATH_INFO: `/v3/projects/5df8290f2eaa4c0dac5d354d0358aeac/users/e25194110c9f4b649ab1894676d025e4/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:34.032905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad269a4-4b99-4ead-b6cb-d0c751f4d2ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df8290f2eaa4c0dac5d354d0358aeac, user_id=e25194110c9f4b649ab1894676d025e4, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:34.036276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad269a4-4b99-4ead-b6cb-d0c751f4d2ff admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:34.050478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 998/2039] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:33 2022] POST /identity/v3/projects => generated 401 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:34.053707 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1042/2040] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:34 2022] PUT /identity/v3/projects/5df8290f2eaa4c0dac5d354d0358aeac/users/e25194110c9f4b649ab1894676d025e4/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:34.056990 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c17f6-16a3-4eec-866e-41f68e53bf5b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:34.065821 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc30ead-1c43-495b-a847-eca16c8d60f1 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:34.066061 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc30ead-1c43-495b-a847-eca16c8d60f1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:34.066412 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc30ead-1c43-495b-a847-eca16c8d60f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:34.095889 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-0cc30ead-1c43-495b-a847-eca16c8d60f1 None None] MFA Rules not processed for user `e25194110c9f4b649ab1894676d025e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:34.097121 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0cc30ead-1c43-495b-a847-eca16c8d60f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:34.124303 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:34.161639 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c17f6-16a3-4eec-866e-41f68e53bf5b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:34.167518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:34.167805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0519de-b706-4395-b6e9-fa8eb4c65e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:34.168654 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0519de-b706-4395-b6e9-fa8eb4c65e34 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:34.168979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0519de-b706-4395-b6e9-fa8eb4c65e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:34.169226 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0519de-b706-4395-b6e9-fa8eb4c65e34 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:34.169956 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0519de-b706-4395-b6e9-fa8eb4c65e34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:34.171969 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0519de-b706-4395-b6e9-fa8eb4c65e34 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:34.196065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 999/2041] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:34 2022] POST /identity/v3/users => generated 307 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:34.200917 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0519de-b706-4395-b6e9-fa8eb4c65e34 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:34.207695 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1043/2042] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:34 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:34.214441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0519de-b706-4395-b6e9-fa8eb4c65e34 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:34.219754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc30ead-1c43-495b-a847-eca16c8d60f1 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:34.221270 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0cc30ead-1c43-495b-a847-eca16c8d60f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:34.221390 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a78e0-74b1-44fc-aefb-e2090c5cd0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:34.221954 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a78e0-74b1-44fc-aefb-e2090c5cd0f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:34.222292 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a78e0-74b1-44fc-aefb-e2090c5cd0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:34.222665 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a78e0-74b1-44fc-aefb-e2090c5cd0f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:34.223400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a78e0-74b1-44fc-aefb-e2090c5cd0f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:34.224970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a78e0-74b1-44fc-aefb-e2090c5cd0f9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:34.238032 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1000/2043] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:34.242691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a78e0-74b1-44fc-aefb-e2090c5cd0f9 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:34.258247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a78e0-74b1-44fc-aefb-e2090c5cd0f9 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:34.281486 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3966c8-b07f-4ae0-89dc-166015c2738c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:34.283108 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3966c8-b07f-4ae0-89dc-166015c2738c admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:34.284712 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3966c8-b07f-4ae0-89dc-166015c2738c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:34.285007 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3966c8-b07f-4ae0-89dc-166015c2738c admin admin] PATH_INFO: `/v3/projects/71dfa91a0e1b4ac9b407963b301c804d/users/c457056a5f284be5b00565d644e68990/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:34.317889 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc30ead-1c43-495b-a847-eca16c8d60f1 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:34.327014 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3966c8-b07f-4ae0-89dc-166015c2738c admin admin] RBAC: Authorizing `identity:create_grant(project_id=71dfa91a0e1b4ac9b407963b301c804d, user_id=c457056a5f284be5b00565d644e68990, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:34.332343 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3966c8-b07f-4ae0-89dc-166015c2738c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:34.335801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce090a-5130-4f25-ac44-43ab6cb0ef33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:34.336552 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce090a-5130-4f25-ac44-43ab6cb0ef33 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:34.336887 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce090a-5130-4f25-ac44-43ab6cb0ef33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:34.338534 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce090a-5130-4f25-ac44-43ab6cb0ef33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:34.362999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1001/2044] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:34 2022] PUT /identity/v3/projects/71dfa91a0e1b4ac9b407963b301c804d/users/c457056a5f284be5b00565d644e68990/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:34.363767 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce090a-5130-4f25-ac44-43ab6cb0ef33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:34.365792 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce090a-5130-4f25-ac44-43ab6cb0ef33 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:34.368172 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3966c8-b07f-4ae0-89dc-166015c2738c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:34.402241 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1044/2045] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:39:34.452565 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce090a-5130-4f25-ac44-43ab6cb0ef33 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:34.454065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-62ce090a-5130-4f25-ac44-43ab6cb0ef33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:34.468711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce090a-5130-4f25-ac44-43ab6cb0ef33 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:34.478078 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3966c8-b07f-4ae0-89dc-166015c2738c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:34.482938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24c67a-2c5f-4d56-8230-5f0af293e0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:34.483611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24c67a-2c5f-4d56-8230-5f0af293e0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:34.483902 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24c67a-2c5f-4d56-8230-5f0af293e0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:34.484183 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24c67a-2c5f-4d56-8230-5f0af293e0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:34.484934 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24c67a-2c5f-4d56-8230-5f0af293e0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:34.486516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24c67a-2c5f-4d56-8230-5f0af293e0e6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:34.496359 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1002/2046] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:34.500540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24c67a-2c5f-4d56-8230-5f0af293e0e6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:34.510564 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24c67a-2c5f-4d56-8230-5f0af293e0e6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:34.515280 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d55cf8-4001-45fb-9e4f-7dba6dbba332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:34.516147 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d55cf8-4001-45fb-9e4f-7dba6dbba332 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:34.516429 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d55cf8-4001-45fb-9e4f-7dba6dbba332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:34.516705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d55cf8-4001-45fb-9e4f-7dba6dbba332 admin admin] PATH_INFO: `/v3/projects/71dfa91a0e1b4ac9b407963b301c804d/users/c457056a5f284be5b00565d644e68990/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:34.518701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d55cf8-4001-45fb-9e4f-7dba6dbba332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71dfa91a0e1b4ac9b407963b301c804d, user_id=c457056a5f284be5b00565d644e68990, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:34.521309 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d55cf8-4001-45fb-9e4f-7dba6dbba332 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:34.527575 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1309b93-b01f-4561-bcdb-27a231a93b54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:34.528252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1309b93-b01f-4561-bcdb-27a231a93b54 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:34.528536 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1309b93-b01f-4561-bcdb-27a231a93b54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:34.528814 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1309b93-b01f-4561-bcdb-27a231a93b54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:34.532264 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1003/2047] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:34 2022] PUT /identity/v3/projects/71dfa91a0e1b4ac9b407963b301c804d/users/c457056a5f284be5b00565d644e68990/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:34.542677 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dbab7-4d49-46d0-babc-547457503434 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:34.542981 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dbab7-4d49-46d0-babc-547457503434 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:34.543266 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dbab7-4d49-46d0-babc-547457503434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:34.575853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-625dbab7-4d49-46d0-babc-547457503434 None None] MFA Rules not processed for user `c457056a5f284be5b00565d644e68990`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:34.577114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-625dbab7-4d49-46d0-babc-547457503434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:34.591894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1309b93-b01f-4561-bcdb-27a231a93b54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:34.593347 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1309b93-b01f-4561-bcdb-27a231a93b54 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:34.602252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1045/2048] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:34 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:39:34.645987 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:34.655212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1004/2049] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:34 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:39:34.663674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1309b93-b01f-4561-bcdb-27a231a93b54 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:34.675568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1309b93-b01f-4561-bcdb-27a231a93b54 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:34.713574 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:34.737724 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f638c-9ce3-46fe-8575-4d672c8a260b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:34.738390 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f638c-9ce3-46fe-8575-4d672c8a260b service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:34.738717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f638c-9ce3-46fe-8575-4d672c8a260b service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:34.739079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f638c-9ce3-46fe-8575-4d672c8a260b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:34.751137 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f638c-9ce3-46fe-8575-4d672c8a260b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:34.752576 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f638c-9ce3-46fe-8575-4d672c8a260b service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:34.770295 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1046/2050] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:34 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:34.849003 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625dbab7-4d49-46d0-babc-547457503434 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:34.859132 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625dbab7-4d49-46d0-babc-547457503434 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:34.865217 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58edbee-bb62-41e2-92ed-bae72823e05f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:34.865894 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58edbee-bb62-41e2-92ed-bae72823e05f service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:34.866184 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58edbee-bb62-41e2-92ed-bae72823e05f service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:34.866455 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58edbee-bb62-41e2-92ed-bae72823e05f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:35.000084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58edbee-bb62-41e2-92ed-bae72823e05f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:35.001560 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58edbee-bb62-41e2-92ed-bae72823e05f service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:35.025763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1005/2051] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:34 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:35.667285 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f638c-9ce3-46fe-8575-4d672c8a260b service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:35.678527 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f638c-9ce3-46fe-8575-4d672c8a260b service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:35.683771 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5f0a5-01ea-4f8c-bc3b-fda77ca0910f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:35.684426 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5f0a5-01ea-4f8c-bc3b-fda77ca0910f service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:35.684707 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5f0a5-01ea-4f8c-bc3b-fda77ca0910f service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:35.684982 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5f0a5-01ea-4f8c-bc3b-fda77ca0910f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:35.687011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-48e5f0a5-01ea-4f8c-bc3b-fda77ca0910f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:35.790825 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5f0a5-01ea-4f8c-bc3b-fda77ca0910f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:35.792323 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5f0a5-01ea-4f8c-bc3b-fda77ca0910f service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:35.802825 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1047/2052] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:37.764015 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58edbee-bb62-41e2-92ed-bae72823e05f service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:37.765455 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c58edbee-bb62-41e2-92ed-bae72823e05f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:37.858043 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58edbee-bb62-41e2-92ed-bae72823e05f service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:37.863154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb4c55-4368-459f-868c-bfbd33ec77bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:37.864064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb4c55-4368-459f-868c-bfbd33ec77bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:37.864352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb4c55-4368-459f-868c-bfbd33ec77bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:37.864666 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb4c55-4368-459f-868c-bfbd33ec77bc admin admin] PATH_INFO: `/v3/users/2e14d892eae64aea8ef952d23cccdff3` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:37.865970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb4c55-4368-459f-868c-bfbd33ec77bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e14d892eae64aea8ef952d23cccdff3)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:37.868055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb4c55-4368-459f-868c-bfbd33ec77bc admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:37.918714 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3abb4c55-4368-459f-868c-bfbd33ec77bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e14d892eae64aea8ef952d23cccdff3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:37.928322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3abb4c55-4368-459f-868c-bfbd33ec77bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e14d892eae64aea8ef952d23cccdff3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:37.936461 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3abb4c55-4368-459f-868c-bfbd33ec77bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e14d892eae64aea8ef952d23cccdff3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:37.954108 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3abb4c55-4368-459f-868c-bfbd33ec77bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e14d892eae64aea8ef952d23cccdff3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:37.954996 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3abb4c55-4368-459f-868c-bfbd33ec77bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e14d892eae64aea8ef952d23cccdff3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:37.965200 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1006/2053] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:37 2022] DELETE /identity/v3/users/2e14d892eae64aea8ef952d23cccdff3 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:37.969941 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b31ac5-f744-4f72-93ad-af4b9ee1f7e2 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:37.970950 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b31ac5-f744-4f72-93ad-af4b9ee1f7e2 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:37.971341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b31ac5-f744-4f72-93ad-af4b9ee1f7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:37.995042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-94b31ac5-f744-4f72-93ad-af4b9ee1f7e2 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:37.996615 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-94b31ac5-f744-4f72-93ad-af4b9ee1f7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:38.063747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1048/2054] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:38.075852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb4c55-4368-459f-868c-bfbd33ec77bc admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:38.077443 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3abb4c55-4368-459f-868c-bfbd33ec77bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:38.100196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:38.146612 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb4c55-4368-459f-868c-bfbd33ec77bc admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:38.150944 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f38c5-bb32-40e3-9780-328a15a526ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:38.151498 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f38c5-bb32-40e3-9780-328a15a526ce service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:38.151762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f38c5-bb32-40e3-9780-328a15a526ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:38.152036 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f38c5-bb32-40e3-9780-328a15a526ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:38.163845 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f38c5-bb32-40e3-9780-328a15a526ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:38.165817 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f38c5-bb32-40e3-9780-328a15a526ce service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:38.179070 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1007/2055] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:38.544972 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b31ac5-f744-4f72-93ad-af4b9ee1f7e2 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:38.550023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-94b31ac5-f744-4f72-93ad-af4b9ee1f7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:38.587985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b31ac5-f744-4f72-93ad-af4b9ee1f7e2 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:38.596981 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db15ef-03d4-4ef5-b4f7-f1c75290e98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:38.598295 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db15ef-03d4-4ef5-b4f7-f1c75290e98e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:38.600771 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db15ef-03d4-4ef5-b4f7-f1c75290e98e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:38.601377 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db15ef-03d4-4ef5-b4f7-f1c75290e98e admin admin] PATH_INFO: `/v3/projects/0849b36882564f989dfb96fd06203e09` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:38.603127 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db15ef-03d4-4ef5-b4f7-f1c75290e98e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0849b36882564f989dfb96fd06203e09)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:38.606932 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db15ef-03d4-4ef5-b4f7-f1c75290e98e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:38.671614 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f38c5-bb32-40e3-9780-328a15a526ce service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:38.673162 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f70f38c5-bb32-40e3-9780-328a15a526ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:38.693760 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f38c5-bb32-40e3-9780-328a15a526ce service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:38.722739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c6db15ef-03d4-4ef5-b4f7-f1c75290e98e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0849b36882564f989dfb96fd06203e09', 'request_id': 'req-c6db15ef-03d4-4ef5-b4f7-f1c75290e98e'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:38.753329 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c6db15ef-03d4-4ef5-b4f7-f1c75290e98e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0849b36882564f989dfb96fd06203e09', 'request_id': 'req-c6db15ef-03d4-4ef5-b4f7-f1c75290e98e'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:38.763009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c6db15ef-03d4-4ef5-b4f7-f1c75290e98e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0849b36882564f989dfb96fd06203e09', 'request_id': 'req-c6db15ef-03d4-4ef5-b4f7-f1c75290e98e'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:38.768156 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:38.795388 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c6db15ef-03d4-4ef5-b4f7-f1c75290e98e admin admin] The token cache is being invalidate because project 0849b36882564f989dfb96fd06203e09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:38.795784 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c6db15ef-03d4-4ef5-b4f7-f1c75290e98e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:38.796584 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c6db15ef-03d4-4ef5-b4f7-f1c75290e98e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:38.802947 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1049/2056] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:38 2022] DELETE /identity/v3/projects/0849b36882564f989dfb96fd06203e09 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:38.807099 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db15ef-03d4-4ef5-b4f7-f1c75290e98e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:38.809893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c6db15ef-03d4-4ef5-b4f7-f1c75290e98e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:38.826818 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1f058-a231-4a54-89f3-29873b59deb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:38.830888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1f058-a231-4a54-89f3-29873b59deb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:38.831364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1f058-a231-4a54-89f3-29873b59deb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:38.833921 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1f058-a231-4a54-89f3-29873b59deb3 admin admin] PATH_INFO: `/v3/users/fb5e15195e724bffb84f70bf8d822f7a` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:38.834386 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1f058-a231-4a54-89f3-29873b59deb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb5e15195e724bffb84f70bf8d822f7a)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:38.838214 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1f058-a231-4a54-89f3-29873b59deb3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:38.862242 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db15ef-03d4-4ef5-b4f7-f1c75290e98e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:38.879306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116dd3ab-0611-48f3-aa57-866d5444ef1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:38.882554 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dd3ab-0611-48f3-aa57-866d5444ef1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:38.883069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dd3ab-0611-48f3-aa57-866d5444ef1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:38.883628 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dd3ab-0611-48f3-aa57-866d5444ef1c admin admin] PATH_INFO: `/v3/users/b1f54ae9af6d4d949df5dfac318967b4` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:38.887063 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116dd3ab-0611-48f3-aa57-866d5444ef1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1f54ae9af6d4d949df5dfac318967b4)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:38.889727 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116dd3ab-0611-48f3-aa57-866d5444ef1c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:38.949832 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3ea1f058-a231-4a54-89f3-29873b59deb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb5e15195e724bffb84f70bf8d822f7a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:38.960986 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3ea1f058-a231-4a54-89f3-29873b59deb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb5e15195e724bffb84f70bf8d822f7a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:38.980871 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-116dd3ab-0611-48f3-aa57-866d5444ef1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1f54ae9af6d4d949df5dfac318967b4'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:38.985654 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3ea1f058-a231-4a54-89f3-29873b59deb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb5e15195e724bffb84f70bf8d822f7a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:38.994021 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-116dd3ab-0611-48f3-aa57-866d5444ef1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1f54ae9af6d4d949df5dfac318967b4'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:38.995198 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-116dd3ab-0611-48f3-aa57-866d5444ef1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1f54ae9af6d4d949df5dfac318967b4'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:39.005172 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3ea1f058-a231-4a54-89f3-29873b59deb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb5e15195e724bffb84f70bf8d822f7a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:39.005997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3ea1f058-a231-4a54-89f3-29873b59deb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb5e15195e724bffb84f70bf8d822f7a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:39.013134 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1008/2057] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:38 2022] DELETE /identity/v3/users/fb5e15195e724bffb84f70bf8d822f7a => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:39.014524 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-116dd3ab-0611-48f3-aa57-866d5444ef1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1f54ae9af6d4d949df5dfac318967b4'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:39.015410 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-116dd3ab-0611-48f3-aa57-866d5444ef1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1f54ae9af6d4d949df5dfac318967b4'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:39.017630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a9cc0-9d6a-4f28-b62c-aec49cf7590e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:39.017929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a9cc0-9d6a-4f28-b62c-aec49cf7590e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:39.018212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a9cc0-9d6a-4f28-b62c-aec49cf7590e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:39.033859 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1050/2058] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:38 2022] DELETE /identity/v3/users/b1f54ae9af6d4d949df5dfac318967b4 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:39.049449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3b5a9cc0-9d6a-4f28-b62c-aec49cf7590e None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:39.050904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3b5a9cc0-9d6a-4f28-b62c-aec49cf7590e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:39.090731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:39.125939 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1009/2059] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:39.142024 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116dd3ab-0611-48f3-aa57-866d5444ef1c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:39.143687 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-116dd3ab-0611-48f3-aa57-866d5444ef1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:39.254285 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116dd3ab-0611-48f3-aa57-866d5444ef1c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:39.260758 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2188996-8cc7-4708-b313-86bd8b26e169 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:39.262113 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2188996-8cc7-4708-b313-86bd8b26e169 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:39.262113 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2188996-8cc7-4708-b313-86bd8b26e169 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:39.262934 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2188996-8cc7-4708-b313-86bd8b26e169 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:39.277271 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2188996-8cc7-4708-b313-86bd8b26e169 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:39.278790 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2188996-8cc7-4708-b313-86bd8b26e169 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:39.289808 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1051/2060] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:39.642615 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a9cc0-9d6a-4f28-b62c-aec49cf7590e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:39.644332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3b5a9cc0-9d6a-4f28-b62c-aec49cf7590e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:39.658412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a9cc0-9d6a-4f28-b62c-aec49cf7590e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:39.666568 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13416312-e4b7-42f9-91bf-09d2832e1945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:39.667509 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13416312-e4b7-42f9-91bf-09d2832e1945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:39.667945 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13416312-e4b7-42f9-91bf-09d2832e1945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:39.668298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13416312-e4b7-42f9-91bf-09d2832e1945 admin admin] PATH_INFO: `/v3/users/90f539bd531548f09c8c40e909ca356e` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:39.668404 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2188996-8cc7-4708-b313-86bd8b26e169 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:39.670445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13416312-e4b7-42f9-91bf-09d2832e1945 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90f539bd531548f09c8c40e909ca356e)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:39.670562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f2188996-8cc7-4708-b313-86bd8b26e169 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:39.672761 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13416312-e4b7-42f9-91bf-09d2832e1945 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:39.684498 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2188996-8cc7-4708-b313-86bd8b26e169 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:39.690335 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb523aab-d75d-4955-9a34-2190e4248ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:39.691271 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb523aab-d75d-4955-9a34-2190e4248ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:39.691562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb523aab-d75d-4955-9a34-2190e4248ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:39.691828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb523aab-d75d-4955-9a34-2190e4248ab5 admin admin] PATH_INFO: `/v3/projects/98c76e5b4426488dbaf935e45fd3353e` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:39.693084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb523aab-d75d-4955-9a34-2190e4248ab5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98c76e5b4426488dbaf935e45fd3353e)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:39.695104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb523aab-d75d-4955-9a34-2190e4248ab5 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:39.739352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-13416312-e4b7-42f9-91bf-09d2832e1945 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90f539bd531548f09c8c40e909ca356e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:39.764804 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-13416312-e4b7-42f9-91bf-09d2832e1945 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90f539bd531548f09c8c40e909ca356e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:39.777529 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-13416312-e4b7-42f9-91bf-09d2832e1945 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90f539bd531548f09c8c40e909ca356e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:39.797772 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-13416312-e4b7-42f9-91bf-09d2832e1945 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90f539bd531548f09c8c40e909ca356e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:39.798574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-13416312-e4b7-42f9-91bf-09d2832e1945 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90f539bd531548f09c8c40e909ca356e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:39.806296 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1010/2061] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:39 2022] DELETE /identity/v3/users/90f539bd531548f09c8c40e909ca356e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:39.810000 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13416312-e4b7-42f9-91bf-09d2832e1945 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:39.811561 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-13416312-e4b7-42f9-91bf-09d2832e1945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:39.814771 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-fb523aab-d75d-4955-9a34-2190e4248ab5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98c76e5b4426488dbaf935e45fd3353e', 'request_id': 'req-fb523aab-d75d-4955-9a34-2190e4248ab5'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:39.822560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-fb523aab-d75d-4955-9a34-2190e4248ab5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98c76e5b4426488dbaf935e45fd3353e', 'request_id': 'req-fb523aab-d75d-4955-9a34-2190e4248ab5'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:39.841673 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-fb523aab-d75d-4955-9a34-2190e4248ab5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98c76e5b4426488dbaf935e45fd3353e', 'request_id': 'req-fb523aab-d75d-4955-9a34-2190e4248ab5'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:39.846609 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:39.860846 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-fb523aab-d75d-4955-9a34-2190e4248ab5 admin admin] The token cache is being invalidate because project 98c76e5b4426488dbaf935e45fd3353e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:39.861571 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-fb523aab-d75d-4955-9a34-2190e4248ab5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:39.863029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-fb523aab-d75d-4955-9a34-2190e4248ab5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:39.870747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1052/2062] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:39 2022] DELETE /identity/v3/projects/98c76e5b4426488dbaf935e45fd3353e => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:39.872622 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03803ae1-dd86-4113-a495-49fa203f68a9 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:39.873091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03803ae1-dd86-4113-a495-49fa203f68a9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:39.873542 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03803ae1-dd86-4113-a495-49fa203f68a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:39.888526 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13416312-e4b7-42f9-91bf-09d2832e1945 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:39.895638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c93f4-dc44-4009-80e5-45e0a97b0396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:39.896589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c93f4-dc44-4009-80e5-45e0a97b0396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:39.897024 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c93f4-dc44-4009-80e5-45e0a97b0396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:39.897311 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c93f4-dc44-4009-80e5-45e0a97b0396 admin admin] PATH_INFO: `/v3/projects/482830802c2444638f044e2657862ddb` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:39.898553 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c93f4-dc44-4009-80e5-45e0a97b0396 admin admin] RBAC: Authorizing `identity:delete_project(project_id=482830802c2444638f044e2657862ddb)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:39.899544 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-03803ae1-dd86-4113-a495-49fa203f68a9 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:39.900629 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c93f4-dc44-4009-80e5-45e0a97b0396 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:39.900805 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-03803ae1-dd86-4113-a495-49fa203f68a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:39.912448 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1053/2063] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:39.914149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fd214-1825-478e-955a-cb6a24ea7904 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:39.914428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fd214-1825-478e-955a-cb6a24ea7904 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:39.914745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fd214-1825-478e-955a-cb6a24ea7904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:39.936824 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-3a6fd214-1825-478e-955a-cb6a24ea7904 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:39.938265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3a6fd214-1825-478e-955a-cb6a24ea7904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:39.948844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1054/2064] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:39.949835 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fd214-1825-478e-955a-cb6a24ea7904 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:39.951116 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3a6fd214-1825-478e-955a-cb6a24ea7904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:39.969499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-168c93f4-dc44-4009-80e5-45e0a97b0396 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '482830802c2444638f044e2657862ddb', 'request_id': 'req-168c93f4-dc44-4009-80e5-45e0a97b0396'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:39.978354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:39.980918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-168c93f4-dc44-4009-80e5-45e0a97b0396 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '482830802c2444638f044e2657862ddb', 'request_id': 'req-168c93f4-dc44-4009-80e5-45e0a97b0396'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:39.990527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-168c93f4-dc44-4009-80e5-45e0a97b0396 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '482830802c2444638f044e2657862ddb', 'request_id': 'req-168c93f4-dc44-4009-80e5-45e0a97b0396'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:40.005670 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-168c93f4-dc44-4009-80e5-45e0a97b0396 admin admin] The token cache is being invalidate because project 482830802c2444638f044e2657862ddb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:40.005988 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-168c93f4-dc44-4009-80e5-45e0a97b0396 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:40.006629 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-168c93f4-dc44-4009-80e5-45e0a97b0396 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:40.008542 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1011/2065] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:39 2022] DELETE /identity/v3/projects/482830802c2444638f044e2657862ddb => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:40.010314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fced6b-b4c6-46e9-bf6f-c2a22ddfae1e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:40.010605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fced6b-b4c6-46e9-bf6f-c2a22ddfae1e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:40.010915 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fced6b-b4c6-46e9-bf6f-c2a22ddfae1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:40.022983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fd214-1825-478e-955a-cb6a24ea7904 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:40.030112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-15fced6b-b4c6-46e9-bf6f-c2a22ddfae1e None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:40.031376 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-15fced6b-b4c6-46e9-bf6f-c2a22ddfae1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:40.094060 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f214b3-7eed-423d-a28c-143f8de104b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:40.094753 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f214b3-7eed-423d-a28c-143f8de104b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:40.095056 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f214b3-7eed-423d-a28c-143f8de104b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:40.095320 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f214b3-7eed-423d-a28c-143f8de104b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:40.097489 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d5f214b3-7eed-423d-a28c-143f8de104b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:40.120251 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1012/2066] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:40.122937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fced6b-b4c6-46e9-bf6f-c2a22ddfae1e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:40.124668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-15fced6b-b4c6-46e9-bf6f-c2a22ddfae1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:40.137973 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fced6b-b4c6-46e9-bf6f-c2a22ddfae1e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:40.151712 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fdd93-0147-456b-8bea-994bc2010bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:40.152761 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fdd93-0147-456b-8bea-994bc2010bf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:40.153125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fdd93-0147-456b-8bea-994bc2010bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:40.153472 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fdd93-0147-456b-8bea-994bc2010bf4 admin admin] PATH_INFO: `/v3/users/e25194110c9f4b649ab1894676d025e4` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:40.154852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16fdd93-0147-456b-8bea-994bc2010bf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e25194110c9f4b649ab1894676d025e4)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:40.157818 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16fdd93-0147-456b-8bea-994bc2010bf4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:40.192120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f214b3-7eed-423d-a28c-143f8de104b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:40.194077 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f214b3-7eed-423d-a28c-143f8de104b9 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:40.205171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1055/2067] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:40.206083 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f214b3-7eed-423d-a28c-143f8de104b9 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:40.215689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f214b3-7eed-423d-a28c-143f8de104b9 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:40.217113 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b16fdd93-0147-456b-8bea-994bc2010bf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e25194110c9f4b649ab1894676d025e4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:40.221351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d73b4e-28e6-4d9d-9e2e-b13c37925169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:40.222147 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d73b4e-28e6-4d9d-9e2e-b13c37925169 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:40.222696 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d73b4e-28e6-4d9d-9e2e-b13c37925169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:40.222985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d73b4e-28e6-4d9d-9e2e-b13c37925169 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:40.223867 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d73b4e-28e6-4d9d-9e2e-b13c37925169 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:39:40.224172 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d73b4e-28e6-4d9d-9e2e-b13c37925169 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:40.225782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d73b4e-28e6-4d9d-9e2e-b13c37925169 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:40.227031 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b16fdd93-0147-456b-8bea-994bc2010bf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e25194110c9f4b649ab1894676d025e4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:40.237353 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b16fdd93-0147-456b-8bea-994bc2010bf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e25194110c9f4b649ab1894676d025e4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:40.239151 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1056/2068] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:39:40.244220 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d73b4e-28e6-4d9d-9e2e-b13c37925169 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:40.255517 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d73b4e-28e6-4d9d-9e2e-b13c37925169 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:40.256735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b16fdd93-0147-456b-8bea-994bc2010bf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e25194110c9f4b649ab1894676d025e4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:40.257675 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b16fdd93-0147-456b-8bea-994bc2010bf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e25194110c9f4b649ab1894676d025e4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:40.260615 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ba883-11e6-42b6-86ce-3c93f805490d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:40.261360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ba883-11e6-42b6-86ce-3c93f805490d admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:40.261875 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ba883-11e6-42b6-86ce-3c93f805490d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:40.262158 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ba883-11e6-42b6-86ce-3c93f805490d admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:40.262922 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ba883-11e6-42b6-86ce-3c93f805490d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:40.264958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ba883-11e6-42b6-86ce-3c93f805490d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:40.266156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1013/2069] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:40 2022] DELETE /identity/v3/users/e25194110c9f4b649ab1894676d025e4 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:40.329447 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1057/2070] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:40 2022] POST /identity/v3/projects => generated 367 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:39:40.335203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fdd93-0147-456b-8bea-994bc2010bf4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:40.336927 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b16fdd93-0147-456b-8bea-994bc2010bf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:40.445306 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:40.452422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fdd93-0147-456b-8bea-994bc2010bf4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:40.550219 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750598a-96b6-4da1-bbe4-1c8c0b6a7f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:40.551517 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750598a-96b6-4da1-bbe4-1c8c0b6a7f2e admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:40.553112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750598a-96b6-4da1-bbe4-1c8c0b6a7f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:40.553408 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750598a-96b6-4da1-bbe4-1c8c0b6a7f2e admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:40.554693 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750598a-96b6-4da1-bbe4-1c8c0b6a7f2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:40.563056 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750598a-96b6-4da1-bbe4-1c8c0b6a7f2e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:40.605198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1014/2071] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:40 2022] POST /identity/v3/users => generated 284 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:39:40.609299 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ba883-11e6-42b6-86ce-3c93f805490d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:40.621174 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ba883-11e6-42b6-86ce-3c93f805490d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:40.627119 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b093f-c616-4c3d-a9b6-5cfcaf747579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:40.627902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b093f-c616-4c3d-a9b6-5cfcaf747579 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:40.628193 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b093f-c616-4c3d-a9b6-5cfcaf747579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:40.628475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b093f-c616-4c3d-a9b6-5cfcaf747579 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:40.629230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082b093f-c616-4c3d-a9b6-5cfcaf747579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:40.631171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082b093f-c616-4c3d-a9b6-5cfcaf747579 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:40.639524 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:40.656269 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1058/2072] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:40.657208 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750598a-96b6-4da1-bbe4-1c8c0b6a7f2e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:40.674832 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750598a-96b6-4da1-bbe4-1c8c0b6a7f2e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:40.688091 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76049feb-2a6d-4ee7-9470-6af835dcfd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:40.689897 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76049feb-2a6d-4ee7-9470-6af835dcfd16 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:40.690191 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76049feb-2a6d-4ee7-9470-6af835dcfd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:40.690515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76049feb-2a6d-4ee7-9470-6af835dcfd16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:40.691478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76049feb-2a6d-4ee7-9470-6af835dcfd16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:40.694091 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76049feb-2a6d-4ee7-9470-6af835dcfd16 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:40.728348 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1015/2073] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:40.730024 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b093f-c616-4c3d-a9b6-5cfcaf747579 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:40.734965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76049feb-2a6d-4ee7-9470-6af835dcfd16 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:40.736935 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-76049feb-2a6d-4ee7-9470-6af835dcfd16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:40.747408 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b093f-c616-4c3d-a9b6-5cfcaf747579 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:40.753795 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc16f6b4-2649-43a2-9c44-090acff1b08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:40.755662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16f6b4-2649-43a2-9c44-090acff1b08d admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:40.757457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16f6b4-2649-43a2-9c44-090acff1b08d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:40.758138 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16f6b4-2649-43a2-9c44-090acff1b08d admin admin] PATH_INFO: `/v3/projects/0a9662d1e844401b839ee78652345f97/users/35184312ee7b426db7a94d8d30e2c060/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:40.764141 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76049feb-2a6d-4ee7-9470-6af835dcfd16 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:40.777603 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad2d5c-54a4-46e9-adfe-7a8fe1b9fdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:40.778753 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad2d5c-54a4-46e9-adfe-7a8fe1b9fdd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:40.778970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad2d5c-54a4-46e9-adfe-7a8fe1b9fdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:40.779252 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad2d5c-54a4-46e9-adfe-7a8fe1b9fdd2 admin admin] PATH_INFO: `/v3/projects/cf833e5e5896499391c08df9c0dbf6dd` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:40.781513 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad2d5c-54a4-46e9-adfe-7a8fe1b9fdd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf833e5e5896499391c08df9c0dbf6dd)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:40.783485 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad2d5c-54a4-46e9-adfe-7a8fe1b9fdd2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:40.801653 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc16f6b4-2649-43a2-9c44-090acff1b08d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a9662d1e844401b839ee78652345f97, user_id=35184312ee7b426db7a94d8d30e2c060, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:40.805010 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc16f6b4-2649-43a2-9c44-090acff1b08d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:40.841754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1059/2074] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:40 2022] PUT /identity/v3/projects/0a9662d1e844401b839ee78652345f97/users/35184312ee7b426db7a94d8d30e2c060/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:40.847423 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69ad0e-f46c-41ea-9c85-4b768246eb70 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:40.847723 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69ad0e-f46c-41ea-9c85-4b768246eb70 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:40.848634 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69ad0e-f46c-41ea-9c85-4b768246eb70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:40.886998 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-bd69ad0e-f46c-41ea-9c85-4b768246eb70 None None] MFA Rules not processed for user `35184312ee7b426db7a94d8d30e2c060`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:40.888314 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-bd69ad0e-f46c-41ea-9c85-4b768246eb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:40.891888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1cad2d5c-54a4-46e9-adfe-7a8fe1b9fdd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf833e5e5896499391c08df9c0dbf6dd', 'request_id': 'req-1cad2d5c-54a4-46e9-adfe-7a8fe1b9fdd2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:40.900357 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1cad2d5c-54a4-46e9-adfe-7a8fe1b9fdd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf833e5e5896499391c08df9c0dbf6dd', 'request_id': 'req-1cad2d5c-54a4-46e9-adfe-7a8fe1b9fdd2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:40.924341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1cad2d5c-54a4-46e9-adfe-7a8fe1b9fdd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf833e5e5896499391c08df9c0dbf6dd', 'request_id': 'req-1cad2d5c-54a4-46e9-adfe-7a8fe1b9fdd2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:41.009475 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1cad2d5c-54a4-46e9-adfe-7a8fe1b9fdd2 admin admin] The token cache is being invalidate because project cf833e5e5896499391c08df9c0dbf6dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:41.009762 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1cad2d5c-54a4-46e9-adfe-7a8fe1b9fdd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:41.013352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1cad2d5c-54a4-46e9-adfe-7a8fe1b9fdd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:41.021003 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1016/2075] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:40 2022] DELETE /identity/v3/projects/cf833e5e5896499391c08df9c0dbf6dd => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:41.021385 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1060/2076] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:40 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:41.022192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad2d5c-54a4-46e9-adfe-7a8fe1b9fdd2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:41.025681 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1cad2d5c-54a4-46e9-adfe-7a8fe1b9fdd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:41.025907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69ad0e-f46c-41ea-9c85-4b768246eb70 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:41.111780 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69ad0e-f46c-41ea-9c85-4b768246eb70 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:41.124815 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7040a6d-c6a0-40bb-8a6f-2953e389ce52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:41.126020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7040a6d-c6a0-40bb-8a6f-2953e389ce52 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:41.126020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7040a6d-c6a0-40bb-8a6f-2953e389ce52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:41.128463 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7040a6d-c6a0-40bb-8a6f-2953e389ce52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:41.129503 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7040a6d-c6a0-40bb-8a6f-2953e389ce52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:41.132071 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7040a6d-c6a0-40bb-8a6f-2953e389ce52 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:41.179919 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad2d5c-54a4-46e9-adfe-7a8fe1b9fdd2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:41.186792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c4ee7-da4b-4468-82c5-782d3ad09028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:41.188084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c4ee7-da4b-4468-82c5-782d3ad09028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:41.188084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c4ee7-da4b-4468-82c5-782d3ad09028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:41.188319 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c4ee7-da4b-4468-82c5-782d3ad09028 admin admin] PATH_INFO: `/v3/users/c457056a5f284be5b00565d644e68990` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:41.193569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c4ee7-da4b-4468-82c5-782d3ad09028 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c457056a5f284be5b00565d644e68990)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:41.193569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c4ee7-da4b-4468-82c5-782d3ad09028 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:41.198108 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:41.202255 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1061/2077] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:41 2022] POST /identity/v3/projects => generated 367 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:41.203372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7040a6d-c6a0-40bb-8a6f-2953e389ce52 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:41.206405 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a7040a6d-c6a0-40bb-8a6f-2953e389ce52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:41.258990 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-561c4ee7-da4b-4468-82c5-782d3ad09028 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c457056a5f284be5b00565d644e68990'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:41.268724 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-561c4ee7-da4b-4468-82c5-782d3ad09028 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c457056a5f284be5b00565d644e68990'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:41.278574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-561c4ee7-da4b-4468-82c5-782d3ad09028 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c457056a5f284be5b00565d644e68990'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:41.300736 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-561c4ee7-da4b-4468-82c5-782d3ad09028 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c457056a5f284be5b00565d644e68990'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:41.301604 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-561c4ee7-da4b-4468-82c5-782d3ad09028 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c457056a5f284be5b00565d644e68990'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:41.304214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:41.310095 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1017/2078] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:41 2022] DELETE /identity/v3/users/c457056a5f284be5b00565d644e68990 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:41.313095 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c4ee7-da4b-4468-82c5-782d3ad09028 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:41.313095 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-561c4ee7-da4b-4468-82c5-782d3ad09028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:41.326942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7040a6d-c6a0-40bb-8a6f-2953e389ce52 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:41.414924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c4ee7-da4b-4468-82c5-782d3ad09028 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:41.421232 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6166f15-c1a0-4b14-865f-d889ab650ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:41.422191 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6166f15-c1a0-4b14-865f-d889ab650ff9 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:41.422680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6166f15-c1a0-4b14-865f-d889ab650ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:41.423132 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6166f15-c1a0-4b14-865f-d889ab650ff9 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:41.423976 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6166f15-c1a0-4b14-865f-d889ab650ff9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:41.426167 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6166f15-c1a0-4b14-865f-d889ab650ff9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:41.445938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8816a-4b5d-4230-8054-af8e5cc1987e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:41.447426 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8816a-4b5d-4230-8054-af8e5cc1987e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:41.447708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8816a-4b5d-4230-8054-af8e5cc1987e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:41.448125 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8816a-4b5d-4230-8054-af8e5cc1987e admin admin] PATH_INFO: `/v3/users/0e7899ad3e6a403e8aca5b277068ef35` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:41.449552 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8816a-4b5d-4230-8054-af8e5cc1987e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e7899ad3e6a403e8aca5b277068ef35)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:41.451637 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8816a-4b5d-4230-8054-af8e5cc1987e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:41.475400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1018/2079] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:41 2022] POST /identity/v3/users => generated 290 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:41.476273 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6166f15-c1a0-4b14-865f-d889ab650ff9 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:41.479429 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e6166f15-c1a0-4b14-865f-d889ab650ff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:41.500743 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6166f15-c1a0-4b14-865f-d889ab650ff9 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:41.507362 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78c230-5190-4955-9525-0cbf62e1d4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:41.508598 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78c230-5190-4955-9525-0cbf62e1d4c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:41.509803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78c230-5190-4955-9525-0cbf62e1d4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:41.510158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78c230-5190-4955-9525-0cbf62e1d4c2 admin admin] PATH_INFO: `/v3/projects/5df8290f2eaa4c0dac5d354d0358aeac` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:41.511713 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78c230-5190-4955-9525-0cbf62e1d4c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5df8290f2eaa4c0dac5d354d0358aeac)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:41.514037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78c230-5190-4955-9525-0cbf62e1d4c2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:41.528829 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-87e8816a-4b5d-4230-8054-af8e5cc1987e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e7899ad3e6a403e8aca5b277068ef35'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:41.545154 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-87e8816a-4b5d-4230-8054-af8e5cc1987e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e7899ad3e6a403e8aca5b277068ef35'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:41.545783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-87e8816a-4b5d-4230-8054-af8e5cc1987e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e7899ad3e6a403e8aca5b277068ef35'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:41.570925 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-87e8816a-4b5d-4230-8054-af8e5cc1987e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e7899ad3e6a403e8aca5b277068ef35'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:41.572319 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-87e8816a-4b5d-4230-8054-af8e5cc1987e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e7899ad3e6a403e8aca5b277068ef35'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:41.589526 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1c78c230-5190-4955-9525-0cbf62e1d4c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5df8290f2eaa4c0dac5d354d0358aeac', 'request_id': 'req-1c78c230-5190-4955-9525-0cbf62e1d4c2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:41.598877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1c78c230-5190-4955-9525-0cbf62e1d4c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5df8290f2eaa4c0dac5d354d0358aeac', 'request_id': 'req-1c78c230-5190-4955-9525-0cbf62e1d4c2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:41.605708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1062/2080] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:41 2022] DELETE /identity/v3/users/0e7899ad3e6a403e8aca5b277068ef35 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:41.607684 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8816a-4b5d-4230-8054-af8e5cc1987e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:41.608618 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1c78c230-5190-4955-9525-0cbf62e1d4c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5df8290f2eaa4c0dac5d354d0358aeac', 'request_id': 'req-1c78c230-5190-4955-9525-0cbf62e1d4c2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:41.611474 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-87e8816a-4b5d-4230-8054-af8e5cc1987e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:41.618859 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=78,keystone:DELETE=17,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:41.633123 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1c78c230-5190-4955-9525-0cbf62e1d4c2 admin admin] The token cache is being invalidate because project 5df8290f2eaa4c0dac5d354d0358aeac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:41.634364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1c78c230-5190-4955-9525-0cbf62e1d4c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:41.638741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1c78c230-5190-4955-9525-0cbf62e1d4c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:41.642163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1019/2081] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:41 2022] DELETE /identity/v3/projects/5df8290f2eaa4c0dac5d354d0358aeac => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:41.643465 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f6ddd-7f12-48c9-864c-69bdd4cd8175 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:41.644499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f6ddd-7f12-48c9-864c-69bdd4cd8175 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:41.645056 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f6ddd-7f12-48c9-864c-69bdd4cd8175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:41.677488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-437f6ddd-7f12-48c9-864c-69bdd4cd8175 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:41.678876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-437f6ddd-7f12-48c9-864c-69bdd4cd8175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:41.720889 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8816a-4b5d-4230-8054-af8e5cc1987e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:41.726124 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42a82f-4b58-45ad-bfb9-371ef8bb8fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:41.727841 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42a82f-4b58-45ad-bfb9-371ef8bb8fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:41.728149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42a82f-4b58-45ad-bfb9-371ef8bb8fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:41.728430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42a82f-4b58-45ad-bfb9-371ef8bb8fa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:41.729636 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42a82f-4b58-45ad-bfb9-371ef8bb8fa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:41.731338 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42a82f-4b58-45ad-bfb9-371ef8bb8fa6 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:41.746755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1063/2082] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:41.748145 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5ef48-ba22-4c7f-9376-1d4fd556e407 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:41.748428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5ef48-ba22-4c7f-9376-1d4fd556e407 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:41.748713 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5ef48-ba22-4c7f-9376-1d4fd556e407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:41.770152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-cfa5ef48-ba22-4c7f-9376-1d4fd556e407 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:41.774832 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cfa5ef48-ba22-4c7f-9376-1d4fd556e407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:41.777536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1020/2083] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:41.778683 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f6ddd-7f12-48c9-864c-69bdd4cd8175 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:41.780509 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-437f6ddd-7f12-48c9-864c-69bdd4cd8175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:41.796037 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1064/2084] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:41.796874 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5ef48-ba22-4c7f-9376-1d4fd556e407 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:41.799374 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cfa5ef48-ba22-4c7f-9376-1d4fd556e407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:41.802361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f6ddd-7f12-48c9-864c-69bdd4cd8175 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:41.809668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9e63e-fe2e-490b-ae0b-3cdb48283470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:41.811006 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9e63e-fe2e-490b-ae0b-3cdb48283470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:41.811761 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9e63e-fe2e-490b-ae0b-3cdb48283470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:41.812145 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9e63e-fe2e-490b-ae0b-3cdb48283470 admin admin] PATH_INFO: `/v3/projects/71dfa91a0e1b4ac9b407963b301c804d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:41.813724 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5ef48-ba22-4c7f-9376-1d4fd556e407 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:41.814074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9e63e-fe2e-490b-ae0b-3cdb48283470 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71dfa91a0e1b4ac9b407963b301c804d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:41.818162 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9e63e-fe2e-490b-ae0b-3cdb48283470 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:41.820071 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0f5e5-f84e-4dda-bf37-66aafee167f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:41.821108 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0f5e5-f84e-4dda-bf37-66aafee167f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:41.821422 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0f5e5-f84e-4dda-bf37-66aafee167f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:41.821732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0f5e5-f84e-4dda-bf37-66aafee167f6 admin admin] PATH_INFO: `/v3/projects/426058ff5a0041a5bf371a0c9464e8a6/users/2302181bfe36431fb99b74c85dd03af5/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:41.824759 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:41.843120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0f5e5-f84e-4dda-bf37-66aafee167f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=426058ff5a0041a5bf371a0c9464e8a6, user_id=2302181bfe36431fb99b74c85dd03af5, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:41.846005 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0f5e5-f84e-4dda-bf37-66aafee167f6 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:41.867830 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1065/2085] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:41 2022] PUT /identity/v3/projects/426058ff5a0041a5bf371a0c9464e8a6/users/2302181bfe36431fb99b74c85dd03af5/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:41.869405 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0f5e5-f84e-4dda-bf37-66aafee167f6 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:41.872510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-35d0f5e5-f84e-4dda-bf37-66aafee167f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:41.878009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-16b9e63e-fe2e-490b-ae0b-3cdb48283470 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71dfa91a0e1b4ac9b407963b301c804d', 'request_id': 'req-16b9e63e-fe2e-490b-ae0b-3cdb48283470'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:41.883821 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-16b9e63e-fe2e-490b-ae0b-3cdb48283470 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71dfa91a0e1b4ac9b407963b301c804d', 'request_id': 'req-16b9e63e-fe2e-490b-ae0b-3cdb48283470'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:41.890209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-16b9e63e-fe2e-490b-ae0b-3cdb48283470 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71dfa91a0e1b4ac9b407963b301c804d', 'request_id': 'req-16b9e63e-fe2e-490b-ae0b-3cdb48283470'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:41.904870 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-16b9e63e-fe2e-490b-ae0b-3cdb48283470 admin admin] The token cache is being invalidate because project 71dfa91a0e1b4ac9b407963b301c804d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:41.905551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-16b9e63e-fe2e-490b-ae0b-3cdb48283470 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:41.906126 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-16b9e63e-fe2e-490b-ae0b-3cdb48283470 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:41.908081 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1021/2086] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:41 2022] DELETE /identity/v3/projects/71dfa91a0e1b4ac9b407963b301c804d => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:41.909403 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2c451-0ccd-4c16-aba5-1767a51447ff None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:41.909787 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2c451-0ccd-4c16-aba5-1767a51447ff None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:41.910137 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2c451-0ccd-4c16-aba5-1767a51447ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:41.933809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3eb2c451-0ccd-4c16-aba5-1767a51447ff None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:41.933809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3eb2c451-0ccd-4c16-aba5-1767a51447ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:41.952680 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0f5e5-f84e-4dda-bf37-66aafee167f6 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:42.024308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1022/2087] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:42.025487 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2c451-0ccd-4c16-aba5-1767a51447ff None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:42.027030 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3eb2c451-0ccd-4c16-aba5-1767a51447ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:42.046229 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2c451-0ccd-4c16-aba5-1767a51447ff None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:42.048087 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f03c95-5555-4025-a6ab-d576e88fe9b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:42.048947 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f03c95-5555-4025-a6ab-d576e88fe9b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:42.049328 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f03c95-5555-4025-a6ab-d576e88fe9b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:42.049607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f03c95-5555-4025-a6ab-d576e88fe9b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:42.051111 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a9921-c5c2-4c3e-94ed-0887a1786783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:42.051399 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-71f03c95-5555-4025-a6ab-d576e88fe9b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:42.051789 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a9921-c5c2-4c3e-94ed-0887a1786783 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:42.052078 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a9921-c5c2-4c3e-94ed-0887a1786783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:42.052379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a9921-c5c2-4c3e-94ed-0887a1786783 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:42.053157 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a9921-c5c2-4c3e-94ed-0887a1786783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:42.054707 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a9921-c5c2-4c3e-94ed-0887a1786783 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:42.066280 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f03c95-5555-4025-a6ab-d576e88fe9b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:42.067296 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1023/2088] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:42.067775 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f03c95-5555-4025-a6ab-d576e88fe9b1 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:42.072945 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5028a7b-6b86-4b29-a5cd-7069d8027c7f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:42.073777 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5028a7b-6b86-4b29-a5cd-7069d8027c7f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:42.073777 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5028a7b-6b86-4b29-a5cd-7069d8027c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:42.078701 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1066/2089] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:42.088936 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f03c95-5555-4025-a6ab-d576e88fe9b1 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:42.097810 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e5028a7b-6b86-4b29-a5cd-7069d8027c7f None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:42.099763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e5028a7b-6b86-4b29-a5cd-7069d8027c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:42.105026 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f03c95-5555-4025-a6ab-d576e88fe9b1 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:42.112890 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647db79-fc87-4359-b913-1c9ce64f0cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:42.113711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647db79-fc87-4359-b913-1c9ce64f0cbb admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:42.114067 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647db79-fc87-4359-b913-1c9ce64f0cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:42.114416 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647db79-fc87-4359-b913-1c9ce64f0cbb admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:42.115354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647db79-fc87-4359-b913-1c9ce64f0cbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:39:42.116034 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647db79-fc87-4359-b913-1c9ce64f0cbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:42.119472 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647db79-fc87-4359-b913-1c9ce64f0cbb admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:42.130242 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1024/2090] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:42.131774 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5028a7b-6b86-4b29-a5cd-7069d8027c7f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:42.138547 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:42.145437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5028a7b-6b86-4b29-a5cd-7069d8027c7f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:42.155130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5848d-940c-46c1-bd1d-f0bafb6f9fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:42.156419 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5848d-940c-46c1-bd1d-f0bafb6f9fc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:42.156991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5848d-940c-46c1-bd1d-f0bafb6f9fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:42.158259 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5848d-940c-46c1-bd1d-f0bafb6f9fc3 admin admin] PATH_INFO: `/v3/projects/426058ff5a0041a5bf371a0c9464e8a6/users/2302181bfe36431fb99b74c85dd03af5/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:42.161285 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5848d-940c-46c1-bd1d-f0bafb6f9fc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=426058ff5a0041a5bf371a0c9464e8a6, user_id=2302181bfe36431fb99b74c85dd03af5, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:42.165653 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5848d-940c-46c1-bd1d-f0bafb6f9fc3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:42.186743 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1067/2092] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:39:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:39:42.198373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff9695-4a8f-4695-96a9-904da4f426c0 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:42.198373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff9695-4a8f-4695-96a9-904da4f426c0 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:42.198373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff9695-4a8f-4695-96a9-904da4f426c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:42.200011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1025/2092] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:42 2022] PUT /identity/v3/projects/426058ff5a0041a5bf371a0c9464e8a6/users/2302181bfe36431fb99b74c85dd03af5/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:42.202113 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddfaf3-33e0-40ef-8278-d0e8c2c1b1d7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:42.202726 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddfaf3-33e0-40ef-8278-d0e8c2c1b1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:42.203218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddfaf3-33e0-40ef-8278-d0e8c2c1b1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:42.244253 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5aff9695-4a8f-4695-96a9-904da4f426c0 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:42.245314 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5aff9695-4a8f-4695-96a9-904da4f426c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:42.280129 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-72ddfaf3-33e0-40ef-8278-d0e8c2c1b1d7 None None] MFA Rules not processed for user `2302181bfe36431fb99b74c85dd03af5`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:42.283313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-72ddfaf3-33e0-40ef-8278-d0e8c2c1b1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:42.394712 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1026/2093] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:42 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:42.394712 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ddfaf3-33e0-40ef-8278-d0e8c2c1b1d7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:42.412323 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1068/2094] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:42.416592 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ddfaf3-33e0-40ef-8278-d0e8c2c1b1d7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:42.417827 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff9695-4a8f-4695-96a9-904da4f426c0 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:42.420123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5aff9695-4a8f-4695-96a9-904da4f426c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:42.429305 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5dfd9c-898a-46d5-aec4-fe9581db10bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:42.431344 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5dfd9c-898a-46d5-aec4-fe9581db10bf admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:42.432936 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5dfd9c-898a-46d5-aec4-fe9581db10bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:42.433516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5dfd9c-898a-46d5-aec4-fe9581db10bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:42.434548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5dfd9c-898a-46d5-aec4-fe9581db10bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:42.440468 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5dfd9c-898a-46d5-aec4-fe9581db10bf admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:42.474639 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:42.490583 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1027/2095] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:42 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:42.496070 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5dfd9c-898a-46d5-aec4-fe9581db10bf admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:42.505522 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff9695-4a8f-4695-96a9-904da4f426c0 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:42.565435 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff063e-c8ea-4119-a163-23187f586f44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:42.565687 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff063e-c8ea-4119-a163-23187f586f44 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:42.565948 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff063e-c8ea-4119-a163-23187f586f44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:42.566364 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff063e-c8ea-4119-a163-23187f586f44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:42.574827 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5dfd9c-898a-46d5-aec4-fe9581db10bf admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:42.580388 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd82c271-094a-4892-a754-327c3800594f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:42.581158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82c271-094a-4892-a754-327c3800594f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:42.581456 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82c271-094a-4892-a754-327c3800594f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:42.581740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82c271-094a-4892-a754-327c3800594f admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:42.582491 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd82c271-094a-4892-a754-327c3800594f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:39:42.582865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd82c271-094a-4892-a754-327c3800594f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:42.584545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd82c271-094a-4892-a754-327c3800594f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:42.602161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1028/2096] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:39:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:39:42.602161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd82c271-094a-4892-a754-327c3800594f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:42.614740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd82c271-094a-4892-a754-327c3800594f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:42.620848 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7703e0-322b-4f75-a62e-60c74251b212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:42.622063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7703e0-322b-4f75-a62e-60c74251b212 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:42.622833 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7703e0-322b-4f75-a62e-60c74251b212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:42.623299 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7703e0-322b-4f75-a62e-60c74251b212 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:42.624498 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7703e0-322b-4f75-a62e-60c74251b212 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:42.640888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7703e0-322b-4f75-a62e-60c74251b212 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:42.646781 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff063e-c8ea-4119-a163-23187f586f44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:42.648242 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff063e-c8ea-4119-a163-23187f586f44 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:42.663140 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1069/2097] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:42 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:42.674719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff063e-c8ea-4119-a163-23187f586f44 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:42.693445 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff063e-c8ea-4119-a163-23187f586f44 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:42.694119 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1029/2098] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:42 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:42.695458 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7703e0-322b-4f75-a62e-60c74251b212 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:42.701839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051df32-f5f0-4b1f-8041-0b52dedb2ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:42.702752 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051df32-f5f0-4b1f-8041-0b52dedb2ac2 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:42.703050 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051df32-f5f0-4b1f-8041-0b52dedb2ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:42.703428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051df32-f5f0-4b1f-8041-0b52dedb2ac2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:42.705097 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051df32-f5f0-4b1f-8041-0b52dedb2ac2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:42.708167 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051df32-f5f0-4b1f-8041-0b52dedb2ac2 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:42.708378 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7703e0-322b-4f75-a62e-60c74251b212 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:42.719573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1ecc6-020a-4a03-848e-3374699f01e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:42.720010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1ecc6-020a-4a03-848e-3374699f01e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:42.720369 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1ecc6-020a-4a03-848e-3374699f01e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:42.720658 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1ecc6-020a-4a03-848e-3374699f01e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:42.721516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1ecc6-020a-4a03-848e-3374699f01e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:42.723787 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1ecc6-020a-4a03-848e-3374699f01e5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:42.747099 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1030/2099] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:42.751071 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1ecc6-020a-4a03-848e-3374699f01e5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:42.756740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d9f1ecc6-020a-4a03-848e-3374699f01e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:42.762161 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1070/2100] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:42 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:42.764165 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051df32-f5f0-4b1f-8041-0b52dedb2ac2 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:42.781527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1ecc6-020a-4a03-848e-3374699f01e5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:42.804713 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:42.851924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9e731-8dd6-47fe-a010-bc33611e415d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:42.852626 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9e731-8dd6-47fe-a010-bc33611e415d service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:42.852912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9e731-8dd6-47fe-a010-bc33611e415d service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:42.853363 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9e731-8dd6-47fe-a010-bc33611e415d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:42.855402 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c7e9e731-8dd6-47fe-a010-bc33611e415d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:42.868451 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051df32-f5f0-4b1f-8041-0b52dedb2ac2 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:42.873821 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecc003-dc52-418f-a328-8cc82981e14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:42.874513 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecc003-dc52-418f-a328-8cc82981e14a admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:42.874828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecc003-dc52-418f-a328-8cc82981e14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:42.875622 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecc003-dc52-418f-a328-8cc82981e14a admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:42.876665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ecc003-dc52-418f-a328-8cc82981e14a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:42.878848 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ecc003-dc52-418f-a328-8cc82981e14a admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:42.891290 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1071/2101] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:42.896419 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecc003-dc52-418f-a328-8cc82981e14a admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:42.918211 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecc003-dc52-418f-a328-8cc82981e14a admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:42.923749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c31d8-9d6d-4c51-bac7-90a40ae5e1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:42.924636 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c31d8-9d6d-4c51-bac7-90a40ae5e1cd admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:42.924870 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9e731-8dd6-47fe-a010-bc33611e415d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:42.925665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c31d8-9d6d-4c51-bac7-90a40ae5e1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:42.926134 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c31d8-9d6d-4c51-bac7-90a40ae5e1cd admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:42.926452 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9e731-8dd6-47fe-a010-bc33611e415d service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:42.927262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c31d8-9d6d-4c51-bac7-90a40ae5e1cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:42.929408 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c31d8-9d6d-4c51-bac7-90a40ae5e1cd admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:42.936551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1031/2102] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:42 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:42.937350 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9e731-8dd6-47fe-a010-bc33611e415d service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:42.939739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:42.949227 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9e731-8dd6-47fe-a010-bc33611e415d service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:42.955677 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1268b-1fb8-43c0-ab3f-97e9a2f453e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:42.958111 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1268b-1fb8-43c0-ab3f-97e9a2f453e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:42.958111 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1268b-1fb8-43c0-ab3f-97e9a2f453e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:42.958111 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1268b-1fb8-43c0-ab3f-97e9a2f453e1 admin admin] PATH_INFO: `/v3/projects/0859e54615fa496781c30362138fcb5d/users/fd5d1f7259034ab79dd3f601c136f415/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:42.967749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1072/2103] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:42 2022] POST /identity/v3/users => generated 293 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:42.971508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c31d8-9d6d-4c51-bac7-90a40ae5e1cd admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:42.987792 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c31d8-9d6d-4c51-bac7-90a40ae5e1cd admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:42.988189 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1268b-1fb8-43c0-ab3f-97e9a2f453e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0859e54615fa496781c30362138fcb5d, user_id=fd5d1f7259034ab79dd3f601c136f415, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:42.994190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1268b-1fb8-43c0-ab3f-97e9a2f453e1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:42.994905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a232de-814a-4972-b809-80150840dcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:42.995615 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a232de-814a-4972-b809-80150840dcd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:42.995903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a232de-814a-4972-b809-80150840dcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:42.996186 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a232de-814a-4972-b809-80150840dcd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:42.996984 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a232de-814a-4972-b809-80150840dcd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:42.999173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a232de-814a-4972-b809-80150840dcd7 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:43.013066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1073/2104] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:39:43.014738 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1032/2105] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:42 2022] PUT /identity/v3/projects/0859e54615fa496781c30362138fcb5d/users/fd5d1f7259034ab79dd3f601c136f415/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:43.025126 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a232de-814a-4972-b809-80150840dcd7 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:43.026496 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1268b-1fb8-43c0-ab3f-97e9a2f453e1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:43.035572 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-99c1268b-1fb8-43c0-ab3f-97e9a2f453e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:43.049808 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a232de-814a-4972-b809-80150840dcd7 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:43.161536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1268b-1fb8-43c0-ab3f-97e9a2f453e1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:43.170121 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920056d5-f49a-410a-be32-c43b906e9f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:43.171879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920056d5-f49a-410a-be32-c43b906e9f0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:43.172430 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920056d5-f49a-410a-be32-c43b906e9f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:43.173319 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920056d5-f49a-410a-be32-c43b906e9f0d admin admin] PATH_INFO: `/v3/projects/7856bd61b5b7433a806fa7d52249e544` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:43.175095 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920056d5-f49a-410a-be32-c43b906e9f0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7856bd61b5b7433a806fa7d52249e544)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:43.175300 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c8a99-c103-4713-9923-43218e86b638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:43.176698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c8a99-c103-4713-9923-43218e86b638 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:43.176996 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c8a99-c103-4713-9923-43218e86b638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:43.177393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c8a99-c103-4713-9923-43218e86b638 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:43.177528 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920056d5-f49a-410a-be32-c43b906e9f0d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:43.178330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c8a99-c103-4713-9923-43218e86b638 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:43.181056 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c8a99-c103-4713-9923-43218e86b638 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:43.200147 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1074/2106] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:43.203390 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b413aa-17dc-4013-971b-64cb31762a31 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:43.204017 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b413aa-17dc-4013-971b-64cb31762a31 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:43.204017 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b413aa-17dc-4013-971b-64cb31762a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:43.241326 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-920056d5-f49a-410a-be32-c43b906e9f0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7856bd61b5b7433a806fa7d52249e544', 'request_id': 'req-920056d5-f49a-410a-be32-c43b906e9f0d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:43.241326 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:43.247522 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-920056d5-f49a-410a-be32-c43b906e9f0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7856bd61b5b7433a806fa7d52249e544', 'request_id': 'req-920056d5-f49a-410a-be32-c43b906e9f0d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:43.248420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-e8b413aa-17dc-4013-971b-64cb31762a31 None None] MFA Rules not processed for user `fd5d1f7259034ab79dd3f601c136f415`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:43.250813 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e8b413aa-17dc-4013-971b-64cb31762a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:43.255379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-920056d5-f49a-410a-be32-c43b906e9f0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7856bd61b5b7433a806fa7d52249e544', 'request_id': 'req-920056d5-f49a-410a-be32-c43b906e9f0d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:43.283513 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-920056d5-f49a-410a-be32-c43b906e9f0d admin admin] The token cache is being invalidate because project 7856bd61b5b7433a806fa7d52249e544 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:43.284115 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-920056d5-f49a-410a-be32-c43b906e9f0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:43.286197 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-920056d5-f49a-410a-be32-c43b906e9f0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:43.298429 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1033/2107] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:43 2022] DELETE /identity/v3/projects/7856bd61b5b7433a806fa7d52249e544 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:43.298429 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920056d5-f49a-410a-be32-c43b906e9f0d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:43.298771 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-920056d5-f49a-410a-be32-c43b906e9f0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:43.391783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1075/2108] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:43.407113 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b413aa-17dc-4013-971b-64cb31762a31 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:43.410631 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e8b413aa-17dc-4013-971b-64cb31762a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:43.412134 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920056d5-f49a-410a-be32-c43b906e9f0d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:43.425555 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa6e3a-63a6-4ee5-b5cf-1d1bb66aa55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:43.426507 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa6e3a-63a6-4ee5-b5cf-1d1bb66aa55d admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:43.426806 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa6e3a-63a6-4ee5-b5cf-1d1bb66aa55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:43.428529 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa6e3a-63a6-4ee5-b5cf-1d1bb66aa55d admin admin] PATH_INFO: `/v3/projects/edef03f0b3ea41eaa5246310071b9b60/users/a5fafca2cb3440a58692934ce3ba2a5b/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:43.451373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b413aa-17dc-4013-971b-64cb31762a31 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:43.458002 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4211ab08-4f58-4477-8a90-99a38bd85022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:43.458817 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211ab08-4f58-4477-8a90-99a38bd85022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:43.459105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211ab08-4f58-4477-8a90-99a38bd85022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:43.459381 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211ab08-4f58-4477-8a90-99a38bd85022 admin admin] PATH_INFO: `/v3/users/1feb45ddd74548b6bdef01111161c608` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:43.460992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4211ab08-4f58-4477-8a90-99a38bd85022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1feb45ddd74548b6bdef01111161c608)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:43.465022 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa6e3a-63a6-4ee5-b5cf-1d1bb66aa55d admin admin] RBAC: Authorizing `identity:create_grant(project_id=edef03f0b3ea41eaa5246310071b9b60, user_id=a5fafca2cb3440a58692934ce3ba2a5b, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:43.467289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4211ab08-4f58-4477-8a90-99a38bd85022 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:43.468097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa6e3a-63a6-4ee5-b5cf-1d1bb66aa55d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:43.506955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1034/2109] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:43 2022] PUT /identity/v3/projects/edef03f0b3ea41eaa5246310071b9b60/users/a5fafca2cb3440a58692934ce3ba2a5b/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:43.507829 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa6e3a-63a6-4ee5-b5cf-1d1bb66aa55d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:43.515800 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e5fa6e3a-63a6-4ee5-b5cf-1d1bb66aa55d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:43.598949 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4211ab08-4f58-4477-8a90-99a38bd85022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1feb45ddd74548b6bdef01111161c608'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:43.609658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4211ab08-4f58-4477-8a90-99a38bd85022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1feb45ddd74548b6bdef01111161c608'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:43.610714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4211ab08-4f58-4477-8a90-99a38bd85022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1feb45ddd74548b6bdef01111161c608'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:43.617577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:43.639289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4211ab08-4f58-4477-8a90-99a38bd85022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1feb45ddd74548b6bdef01111161c608'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:43.639957 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa6e3a-63a6-4ee5-b5cf-1d1bb66aa55d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:43.640072 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4211ab08-4f58-4477-8a90-99a38bd85022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1feb45ddd74548b6bdef01111161c608'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:43.646932 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8ca23-80ba-4861-b880-375ed73e8ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:43.647885 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8ca23-80ba-4861-b880-375ed73e8ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:43.648805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8ca23-80ba-4861-b880-375ed73e8ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:43.649801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8ca23-80ba-4861-b880-375ed73e8ee1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:43.650711 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d8ca23-80ba-4861-b880-375ed73e8ee1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:43.653893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d8ca23-80ba-4861-b880-375ed73e8ee1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:43.678276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1076/2110] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:43 2022] DELETE /identity/v3/users/1feb45ddd74548b6bdef01111161c608 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:43.679883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d7640-0372-49fe-8899-052915b1abf1 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:43.680196 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d7640-0372-49fe-8899-052915b1abf1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:43.680632 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d7640-0372-49fe-8899-052915b1abf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:43.695757 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1035/2111] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:43 2022] POST /identity/v3/projects => generated 373 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:43.701890 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8ca23-80ba-4861-b880-375ed73e8ee1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:43.704263 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-13d8ca23-80ba-4861-b880-375ed73e8ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:43.754377 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-e97d7640-0372-49fe-8899-052915b1abf1 None None] MFA Rules not processed for user `a5fafca2cb3440a58692934ce3ba2a5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:43.756186 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e97d7640-0372-49fe-8899-052915b1abf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:43.783529 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:43.811711 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8ca23-80ba-4861-b880-375ed73e8ee1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:43.821630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666c186-e3d4-4f56-b230-e33481af19bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:43.822555 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666c186-e3d4-4f56-b230-e33481af19bb admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:43.823945 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666c186-e3d4-4f56-b230-e33481af19bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:43.824279 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666c186-e3d4-4f56-b230-e33481af19bb admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:43.825641 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c666c186-e3d4-4f56-b230-e33481af19bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:43.846992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c666c186-e3d4-4f56-b230-e33481af19bb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:43.892057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1077/2112] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:43.892950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1036/2113] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:43 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:43.901242 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d7640-0372-49fe-8899-052915b1abf1 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:43.906384 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666c186-e3d4-4f56-b230-e33481af19bb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:43.912481 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c666c186-e3d4-4f56-b230-e33481af19bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:43.926558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d7640-0372-49fe-8899-052915b1abf1 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:43.943064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5906fc3-616c-4d45-b173-eb9427993bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:43.944126 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5906fc3-616c-4d45-b173-eb9427993bd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:43.944639 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5906fc3-616c-4d45-b173-eb9427993bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:43.945121 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5906fc3-616c-4d45-b173-eb9427993bd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:43.946131 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5906fc3-616c-4d45-b173-eb9427993bd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:43.948105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5906fc3-616c-4d45-b173-eb9427993bd1 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:43.977263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1078/2114] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:43.979958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5906fc3-616c-4d45-b173-eb9427993bd1 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:44.003173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5906fc3-616c-4d45-b173-eb9427993bd1 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:44.019517 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b60ff-68ff-4178-a840-27879859f71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:44.020905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b60ff-68ff-4178-a840-27879859f71f admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:44.021330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b60ff-68ff-4178-a840-27879859f71f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:44.021612 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b60ff-68ff-4178-a840-27879859f71f admin admin] PATH_INFO: `/v3/projects/7e17f4d9498842f882e2664c451499af/users/d35c5d8167014ed5ab1f5b8d3eb0b623/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:44.044002 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b60ff-68ff-4178-a840-27879859f71f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e17f4d9498842f882e2664c451499af, user_id=d35c5d8167014ed5ab1f5b8d3eb0b623, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:44.047414 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b60ff-68ff-4178-a840-27879859f71f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:44.060986 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666c186-e3d4-4f56-b230-e33481af19bb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:44.068737 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1079/2115] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:43 2022] PUT /identity/v3/projects/7e17f4d9498842f882e2664c451499af/users/d35c5d8167014ed5ab1f5b8d3eb0b623/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:44.073779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b60ff-68ff-4178-a840-27879859f71f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:44.120457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1934f1c-409a-4975-9278-cb9ad9cf7fae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:44.120939 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1934f1c-409a-4975-9278-cb9ad9cf7fae service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:44.121247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1934f1c-409a-4975-9278-cb9ad9cf7fae service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:44.121564 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1934f1c-409a-4975-9278-cb9ad9cf7fae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:44.208927 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b60ff-68ff-4178-a840-27879859f71f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:44.214693 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c574d-c490-489e-842b-9baf226055d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:44.215669 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c574d-c490-489e-842b-9baf226055d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:44.216098 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c574d-c490-489e-842b-9baf226055d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:44.216516 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c574d-c490-489e-842b-9baf226055d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:44.218883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c574d-c490-489e-842b-9baf226055d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:44.221373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c574d-c490-489e-842b-9baf226055d6 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:44.240344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1080/2116] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:44.247623 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c574d-c490-489e-842b-9baf226055d6 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:44.255548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1934f1c-409a-4975-9278-cb9ad9cf7fae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:44.256966 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1934f1c-409a-4975-9278-cb9ad9cf7fae service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:44.269408 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c574d-c490-489e-842b-9baf226055d6 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:44.277273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b753f-41c1-445f-bd2d-e1829ffd204c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:44.278393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b753f-41c1-445f-bd2d-e1829ffd204c admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:44.278833 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b753f-41c1-445f-bd2d-e1829ffd204c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:44.279209 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b753f-41c1-445f-bd2d-e1829ffd204c admin admin] PATH_INFO: `/v3/projects/7e17f4d9498842f882e2664c451499af/users/d35c5d8167014ed5ab1f5b8d3eb0b623/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:44.281555 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b753f-41c1-445f-bd2d-e1829ffd204c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e17f4d9498842f882e2664c451499af, user_id=d35c5d8167014ed5ab1f5b8d3eb0b623, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:44.284484 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b753f-41c1-445f-bd2d-e1829ffd204c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:44.287446 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1037/2117] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:44.297380 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1081/2118] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:44 2022] PUT /identity/v3/projects/7e17f4d9498842f882e2664c451499af/users/d35c5d8167014ed5ab1f5b8d3eb0b623/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:44.308200 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd8766-94ff-49c2-bb9c-a8e62f4b2d00 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:44.308200 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd8766-94ff-49c2-bb9c-a8e62f4b2d00 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:44.308480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd8766-94ff-49c2-bb9c-a8e62f4b2d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:44.317759 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b753f-41c1-445f-bd2d-e1829ffd204c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:44.320390 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-708b753f-41c1-445f-bd2d-e1829ffd204c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:44.352022 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-22fd8766-94ff-49c2-bb9c-a8e62f4b2d00 None None] MFA Rules not processed for user `d35c5d8167014ed5ab1f5b8d3eb0b623`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:44.354993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-22fd8766-94ff-49c2-bb9c-a8e62f4b2d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:44.376138 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:44.435122 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:44.471278 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b753f-41c1-445f-bd2d-e1829ffd204c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:44.484195 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b933f-11ca-4409-bdd0-2e8190400899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:44.489187 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b933f-11ca-4409-bdd0-2e8190400899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:44.489549 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b933f-11ca-4409-bdd0-2e8190400899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:44.489905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b933f-11ca-4409-bdd0-2e8190400899 admin admin] PATH_INFO: `/v3/users/a5fafca2cb3440a58692934ce3ba2a5b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:44.492178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b933f-11ca-4409-bdd0-2e8190400899 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5fafca2cb3440a58692934ce3ba2a5b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:44.494997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1038/2119] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:44 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:44.496711 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd8766-94ff-49c2-bb9c-a8e62f4b2d00 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:44.499169 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b933f-11ca-4409-bdd0-2e8190400899 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:44.500540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-22fd8766-94ff-49c2-bb9c-a8e62f4b2d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:44.516247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd8766-94ff-49c2-bb9c-a8e62f4b2d00 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:44.523536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9ef38-ed80-4c0c-90a5-9ae371bc98dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:44.524758 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9ef38-ed80-4c0c-90a5-9ae371bc98dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:44.525039 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9ef38-ed80-4c0c-90a5-9ae371bc98dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:44.525391 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9ef38-ed80-4c0c-90a5-9ae371bc98dd admin admin] PATH_INFO: `/v3/projects/7ca4ec716e8748a081cf1797a42fedd4` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:44.526851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9ef38-ed80-4c0c-90a5-9ae371bc98dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ca4ec716e8748a081cf1797a42fedd4)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:44.528768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9ef38-ed80-4c0c-90a5-9ae371bc98dd admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:44.568233 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f41b933f-11ca-4409-bdd0-2e8190400899 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5fafca2cb3440a58692934ce3ba2a5b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:44.583721 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f41b933f-11ca-4409-bdd0-2e8190400899 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5fafca2cb3440a58692934ce3ba2a5b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:44.584508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f41b933f-11ca-4409-bdd0-2e8190400899 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5fafca2cb3440a58692934ce3ba2a5b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:44.602848 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f41b933f-11ca-4409-bdd0-2e8190400899 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5fafca2cb3440a58692934ce3ba2a5b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:44.603819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-60b9ef38-ed80-4c0c-90a5-9ae371bc98dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ca4ec716e8748a081cf1797a42fedd4', 'request_id': 'req-60b9ef38-ed80-4c0c-90a5-9ae371bc98dd'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:44.604080 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f41b933f-11ca-4409-bdd0-2e8190400899 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5fafca2cb3440a58692934ce3ba2a5b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:44.610522 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-60b9ef38-ed80-4c0c-90a5-9ae371bc98dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ca4ec716e8748a081cf1797a42fedd4', 'request_id': 'req-60b9ef38-ed80-4c0c-90a5-9ae371bc98dd'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:44.618399 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1082/2120] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:44 2022] DELETE /identity/v3/users/a5fafca2cb3440a58692934ce3ba2a5b => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:44.619381 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b933f-11ca-4409-bdd0-2e8190400899 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:44.620756 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f41b933f-11ca-4409-bdd0-2e8190400899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:44.627971 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-60b9ef38-ed80-4c0c-90a5-9ae371bc98dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ca4ec716e8748a081cf1797a42fedd4', 'request_id': 'req-60b9ef38-ed80-4c0c-90a5-9ae371bc98dd'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:44.643373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-60b9ef38-ed80-4c0c-90a5-9ae371bc98dd admin admin] The token cache is being invalidate because project 7ca4ec716e8748a081cf1797a42fedd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:44.643828 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-60b9ef38-ed80-4c0c-90a5-9ae371bc98dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:44.644640 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-60b9ef38-ed80-4c0c-90a5-9ae371bc98dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:44.651564 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1039/2121] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:44 2022] DELETE /identity/v3/projects/7ca4ec716e8748a081cf1797a42fedd4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:44.651564 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855d88e-eaa6-47c8-9469-8cf249a1678a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:44.651564 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855d88e-eaa6-47c8-9469-8cf249a1678a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:44.651564 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855d88e-eaa6-47c8-9469-8cf249a1678a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:44.669986 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9855d88e-eaa6-47c8-9469-8cf249a1678a None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:44.675808 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9855d88e-eaa6-47c8-9469-8cf249a1678a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:44.705646 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b933f-11ca-4409-bdd0-2e8190400899 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:44.713305 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f10cb7-7e78-47a4-9202-f5ad47c20e92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:44.713305 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f10cb7-7e78-47a4-9202-f5ad47c20e92 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:44.713305 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f10cb7-7e78-47a4-9202-f5ad47c20e92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:44.713305 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f10cb7-7e78-47a4-9202-f5ad47c20e92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:44.719174 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e5f10cb7-7e78-47a4-9202-f5ad47c20e92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:44.767329 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1040/2122] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:44.768096 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3454d-aed3-468d-923c-132b33be588c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:44.768096 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3454d-aed3-468d-923c-132b33be588c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:44.770850 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3454d-aed3-468d-923c-132b33be588c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:44.799203 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:44.806022 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0db3454d-aed3-468d-923c-132b33be588c None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:44.807784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0db3454d-aed3-468d-923c-132b33be588c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:44.813153 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f10cb7-7e78-47a4-9202-f5ad47c20e92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:44.814947 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f10cb7-7e78-47a4-9202-f5ad47c20e92 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:44.830845 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1083/2123] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:44.831643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1041/2124] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:44.831850 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f10cb7-7e78-47a4-9202-f5ad47c20e92 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:44.840559 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e5f10cb7-7e78-47a4-9202-f5ad47c20e92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:44.841216 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6852f-a81c-4538-b787-0db85b3598dd None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:44.841610 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6852f-a81c-4538-b787-0db85b3598dd None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:44.841989 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6852f-a81c-4538-b787-0db85b3598dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:44.884405 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-02a6852f-a81c-4538-b787-0db85b3598dd None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:44.885982 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-02a6852f-a81c-4538-b787-0db85b3598dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:44.911497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1042/2125] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:44.916424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6852f-a81c-4538-b787-0db85b3598dd None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:44.925589 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f10cb7-7e78-47a4-9202-f5ad47c20e92 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:44.934754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9ea92-ee17-43fd-bdef-35e9f940aaf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:44.935424 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9ea92-ee17-43fd-bdef-35e9f940aaf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:44.935709 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9ea92-ee17-43fd-bdef-35e9f940aaf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:44.936007 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6852f-a81c-4538-b787-0db85b3598dd None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:44.936956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9ea92-ee17-43fd-bdef-35e9f940aaf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:44.948803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff2e86-1352-4782-b68a-e7fef1c852c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:44.951880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff2e86-1352-4782-b68a-e7fef1c852c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:44.953421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff2e86-1352-4782-b68a-e7fef1c852c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:44.953748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff2e86-1352-4782-b68a-e7fef1c852c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:44.956037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff2e86-1352-4782-b68a-e7fef1c852c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:39:44.956483 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff2e86-1352-4782-b68a-e7fef1c852c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:44.961027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff2e86-1352-4782-b68a-e7fef1c852c9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:44.965630 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be9ea92-ee17-43fd-bdef-35e9f940aaf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:44.967477 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be9ea92-ee17-43fd-bdef-35e9f940aaf9 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:44.992022 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1043/2126] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:39:44.992331 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1084/2127] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:44.996199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff2e86-1352-4782-b68a-e7fef1c852c9 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:45.019594 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff2e86-1352-4782-b68a-e7fef1c852c9 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:45.025543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f17b29-6c1a-4960-a2a8-128814e098d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:45.026515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f17b29-6c1a-4960-a2a8-128814e098d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:45.026905 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f17b29-6c1a-4960-a2a8-128814e098d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:45.027390 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f17b29-6c1a-4960-a2a8-128814e098d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:45.028299 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f17b29-6c1a-4960-a2a8-128814e098d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:45.030397 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f17b29-6c1a-4960-a2a8-128814e098d3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:45.092164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1044/2128] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:44 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:45.097316 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9ea92-ee17-43fd-bdef-35e9f940aaf9 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:45.130094 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9ea92-ee17-43fd-bdef-35e9f940aaf9 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:45.257313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27098644-7e45-41f6-b498-d3fb65427f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:45.258016 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27098644-7e45-41f6-b498-d3fb65427f33 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:45.259592 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27098644-7e45-41f6-b498-d3fb65427f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:45.260034 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27098644-7e45-41f6-b498-d3fb65427f33 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:45.260809 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27098644-7e45-41f6-b498-d3fb65427f33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:45.264328 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27098644-7e45-41f6-b498-d3fb65427f33 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:45.312447 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1085/2129] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:45 2022] POST /identity/v3/users => generated 287 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:45.318070 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f17b29-6c1a-4960-a2a8-128814e098d3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:45.338562 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f17b29-6c1a-4960-a2a8-128814e098d3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:45.360781 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e24eb6-ffb4-4846-a04e-46288afe8c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:45.360781 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e24eb6-ffb4-4846-a04e-46288afe8c17 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:45.360781 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e24eb6-ffb4-4846-a04e-46288afe8c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:45.360781 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e24eb6-ffb4-4846-a04e-46288afe8c17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:45.361219 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e24eb6-ffb4-4846-a04e-46288afe8c17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:45.361219 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e24eb6-ffb4-4846-a04e-46288afe8c17 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:45.361501 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:45.380195 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1045/2130] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:45.385499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27098644-7e45-41f6-b498-d3fb65427f33 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:45.406079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27098644-7e45-41f6-b498-d3fb65427f33 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:45.420253 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d84e5f-b0ef-4b70-bc4c-9b9ff93e291d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:45.421687 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d84e5f-b0ef-4b70-bc4c-9b9ff93e291d admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:45.422350 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d84e5f-b0ef-4b70-bc4c-9b9ff93e291d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:45.423005 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d84e5f-b0ef-4b70-bc4c-9b9ff93e291d admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:45.424393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d84e5f-b0ef-4b70-bc4c-9b9ff93e291d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:45.426469 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d84e5f-b0ef-4b70-bc4c-9b9ff93e291d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:45.470129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1086/2131] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:45.475146 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e24eb6-ffb4-4846-a04e-46288afe8c17 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:45.489018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e24eb6-ffb4-4846-a04e-46288afe8c17 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:45.494604 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c5ecc-5c36-4b23-b6e8-b0f35add3bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:45.495758 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c5ecc-5c36-4b23-b6e8-b0f35add3bb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:45.496136 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c5ecc-5c36-4b23-b6e8-b0f35add3bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:45.496571 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c5ecc-5c36-4b23-b6e8-b0f35add3bb1 admin admin] PATH_INFO: `/v3/projects/fd3b6eefba8a4f5aaeb5519969d9ce25/users/36e6555dbc5e4c7d909d753798cf0f9b/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:45.513737 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c5ecc-5c36-4b23-b6e8-b0f35add3bb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd3b6eefba8a4f5aaeb5519969d9ce25, user_id=36e6555dbc5e4c7d909d753798cf0f9b, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:45.516975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c5ecc-5c36-4b23-b6e8-b0f35add3bb1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:45.536776 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1046/2132] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:45 2022] PUT /identity/v3/projects/fd3b6eefba8a4f5aaeb5519969d9ce25/users/36e6555dbc5e4c7d909d753798cf0f9b/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:45.549308 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d0e92-6b5f-4174-8920-999fc2b18078 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:45.549647 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d0e92-6b5f-4174-8920-999fc2b18078 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:45.549941 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d0e92-6b5f-4174-8920-999fc2b18078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:45.578030 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-cc0d0e92-6b5f-4174-8920-999fc2b18078 None None] MFA Rules not processed for user `36e6555dbc5e4c7d909d753798cf0f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:45.579476 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cc0d0e92-6b5f-4174-8920-999fc2b18078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:45.607722 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c5ecc-5c36-4b23-b6e8-b0f35add3bb1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:45.609449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-485c5ecc-5c36-4b23-b6e8-b0f35add3bb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:45.651848 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:45.655971 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1087/2133] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:45.668246 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d0e92-6b5f-4174-8920-999fc2b18078 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:45.669732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cc0d0e92-6b5f-4174-8920-999fc2b18078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:45.686700 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c5ecc-5c36-4b23-b6e8-b0f35add3bb1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:45.691741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5feac5-1919-4bf4-adea-ec6cf1b7d5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:45.692611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5feac5-1919-4bf4-adea-ec6cf1b7d5d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:45.692906 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5feac5-1919-4bf4-adea-ec6cf1b7d5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:45.693190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5feac5-1919-4bf4-adea-ec6cf1b7d5d7 admin admin] PATH_INFO: `/v3/projects/edef03f0b3ea41eaa5246310071b9b60` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:45.696461 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5feac5-1919-4bf4-adea-ec6cf1b7d5d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edef03f0b3ea41eaa5246310071b9b60)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:45.698600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5feac5-1919-4bf4-adea-ec6cf1b7d5d7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:45.733452 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d0e92-6b5f-4174-8920-999fc2b18078 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:45.746554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dd5feac5-1919-4bf4-adea-ec6cf1b7d5d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edef03f0b3ea41eaa5246310071b9b60', 'request_id': 'req-dd5feac5-1919-4bf4-adea-ec6cf1b7d5d7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:45.752607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dd5feac5-1919-4bf4-adea-ec6cf1b7d5d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edef03f0b3ea41eaa5246310071b9b60', 'request_id': 'req-dd5feac5-1919-4bf4-adea-ec6cf1b7d5d7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:45.761073 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dd5feac5-1919-4bf4-adea-ec6cf1b7d5d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edef03f0b3ea41eaa5246310071b9b60', 'request_id': 'req-dd5feac5-1919-4bf4-adea-ec6cf1b7d5d7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:45.773678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dd5feac5-1919-4bf4-adea-ec6cf1b7d5d7 admin admin] The token cache is being invalidate because project edef03f0b3ea41eaa5246310071b9b60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:45.773973 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dd5feac5-1919-4bf4-adea-ec6cf1b7d5d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:45.775316 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dd5feac5-1919-4bf4-adea-ec6cf1b7d5d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:45.776909 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1047/2134] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:45 2022] DELETE /identity/v3/projects/edef03f0b3ea41eaa5246310071b9b60 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:45.781635 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bd1a9-267b-413a-8798-26c77d3beb8c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:45.781981 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bd1a9-267b-413a-8798-26c77d3beb8c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:45.782211 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bd1a9-267b-413a-8798-26c77d3beb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:45.792200 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15473e-d89e-44c5-b62c-fdd46db21db1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:45.792869 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15473e-d89e-44c5-b62c-fdd46db21db1 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:45.793191 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15473e-d89e-44c5-b62c-fdd46db21db1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:45.793499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15473e-d89e-44c5-b62c-fdd46db21db1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:45.795491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0e15473e-d89e-44c5-b62c-fdd46db21db1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:45.801000 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c64bd1a9-267b-413a-8798-26c77d3beb8c None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:45.802191 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c64bd1a9-267b-413a-8798-26c77d3beb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:45.862708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15473e-d89e-44c5-b62c-fdd46db21db1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:45.864152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15473e-d89e-44c5-b62c-fdd46db21db1 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:45.869899 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1048/2135] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:39:45.878004 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1088/2136] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:39:45.879567 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f57deb-b86a-48d0-8317-9b58d8f0361d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:45.879857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f57deb-b86a-48d0-8317-9b58d8f0361d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:45.880158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f57deb-b86a-48d0-8317-9b58d8f0361d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:45.886966 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:45.901848 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e0f57deb-b86a-48d0-8317-9b58d8f0361d None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:45.903284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e0f57deb-b86a-48d0-8317-9b58d8f0361d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:45.911153 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1049/2137] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:45.916016 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15473e-d89e-44c5-b62c-fdd46db21db1 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:45.926928 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15473e-d89e-44c5-b62c-fdd46db21db1 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:45.932043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2de60-6ce6-418e-a511-923342e396af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:45.932710 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2de60-6ce6-418e-a511-923342e396af admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:45.932998 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2de60-6ce6-418e-a511-923342e396af admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:45.933282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2de60-6ce6-418e-a511-923342e396af admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:45.934014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a2de60-6ce6-418e-a511-923342e396af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:39:45.934329 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a2de60-6ce6-418e-a511-923342e396af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:45.935979 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a2de60-6ce6-418e-a511-923342e396af admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:45.948583 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1089/2138] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:39:45.952951 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f57deb-b86a-48d0-8317-9b58d8f0361d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:45.963280 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f57deb-b86a-48d0-8317-9b58d8f0361d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:45.969275 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e467e8-0f21-4f57-b81d-71ae2723bc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:45.970030 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e467e8-0f21-4f57-b81d-71ae2723bc78 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:45.970557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e467e8-0f21-4f57-b81d-71ae2723bc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:45.970752 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e467e8-0f21-4f57-b81d-71ae2723bc78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:45.971543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e467e8-0f21-4f57-b81d-71ae2723bc78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:45.973565 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e467e8-0f21-4f57-b81d-71ae2723bc78 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:46.005196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1050/2139] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:45 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:46.006317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2de60-6ce6-418e-a511-923342e396af admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:46.020108 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2de60-6ce6-418e-a511-923342e396af admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:46.080572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:46.087384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be6d91-4787-454c-92b1-81e064fd3aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:46.088074 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be6d91-4787-454c-92b1-81e064fd3aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:46.088360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be6d91-4787-454c-92b1-81e064fd3aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:46.088639 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be6d91-4787-454c-92b1-81e064fd3aa5 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:46.089348 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be6d91-4787-454c-92b1-81e064fd3aa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:46.091306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be6d91-4787-454c-92b1-81e064fd3aa5 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:46.112885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1090/2140] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:46 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:46.117455 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e467e8-0f21-4f57-b81d-71ae2723bc78 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:46.128371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e467e8-0f21-4f57-b81d-71ae2723bc78 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:46.133046 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beb5b1f-bb60-490d-a5f9-85ba2103628d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:46.134419 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb5b1f-bb60-490d-a5f9-85ba2103628d admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:46.134633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb5b1f-bb60-490d-a5f9-85ba2103628d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:46.135001 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb5b1f-bb60-490d-a5f9-85ba2103628d admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:46.135667 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beb5b1f-bb60-490d-a5f9-85ba2103628d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:46.137251 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beb5b1f-bb60-490d-a5f9-85ba2103628d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:46.157281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1051/2141] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:39:46.161408 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be6d91-4787-454c-92b1-81e064fd3aa5 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:46.181076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be6d91-4787-454c-92b1-81e064fd3aa5 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:46.186260 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13f87e-0f06-4eee-a160-18140e826bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:46.187462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13f87e-0f06-4eee-a160-18140e826bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:46.187462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13f87e-0f06-4eee-a160-18140e826bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:46.187744 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13f87e-0f06-4eee-a160-18140e826bd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:46.188403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13f87e-0f06-4eee-a160-18140e826bd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:46.190260 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13f87e-0f06-4eee-a160-18140e826bd2 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:46.209723 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1091/2142] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:46.210240 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beb5b1f-bb60-490d-a5f9-85ba2103628d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:46.216540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beb5b1f-bb60-490d-a5f9-85ba2103628d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:46.221316 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80bfc3-280b-4a00-ab7e-bca584af5848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:46.222225 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80bfc3-280b-4a00-ab7e-bca584af5848 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:46.222505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80bfc3-280b-4a00-ab7e-bca584af5848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:46.222811 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80bfc3-280b-4a00-ab7e-bca584af5848 admin admin] PATH_INFO: `/v3/projects/3118a1c283ec4a1cb9a155bc849c15af/users/baf8f24ad16c47eeadd8b6e3a480836d/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:46.240366 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa80bfc3-280b-4a00-ab7e-bca584af5848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3118a1c283ec4a1cb9a155bc849c15af, user_id=baf8f24ad16c47eeadd8b6e3a480836d, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:46.243720 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa80bfc3-280b-4a00-ab7e-bca584af5848 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:46.256221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1052/2143] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:46 2022] PUT /identity/v3/projects/3118a1c283ec4a1cb9a155bc849c15af/users/baf8f24ad16c47eeadd8b6e3a480836d/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:46.266350 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc3c94-3f92-44ba-8bd3-1e26f1dfe6ca None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:46.266763 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc3c94-3f92-44ba-8bd3-1e26f1dfe6ca None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:46.267076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc3c94-3f92-44ba-8bd3-1e26f1dfe6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:46.293983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-dcdc3c94-3f92-44ba-8bd3-1e26f1dfe6ca None None] MFA Rules not processed for user `baf8f24ad16c47eeadd8b6e3a480836d`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:46.295218 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-dcdc3c94-3f92-44ba-8bd3-1e26f1dfe6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:46.365152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1092/2144] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:46.377678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80bfc3-280b-4a00-ab7e-bca584af5848 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:46.379406 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fa80bfc3-280b-4a00-ab7e-bca584af5848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:46.436809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80bfc3-280b-4a00-ab7e-bca584af5848 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:46.440783 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0083c44-b748-4993-a1d2-5d862170968d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:46.441440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0083c44-b748-4993-a1d2-5d862170968d service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:46.441725 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0083c44-b748-4993-a1d2-5d862170968d service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:46.442004 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0083c44-b748-4993-a1d2-5d862170968d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:46.450968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0083c44-b748-4993-a1d2-5d862170968d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:46.452442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0083c44-b748-4993-a1d2-5d862170968d service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:46.462333 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1053/2145] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:46.483972 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc3c94-3f92-44ba-8bd3-1e26f1dfe6ca None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:46.587655 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc3c94-3f92-44ba-8bd3-1e26f1dfe6ca None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:46.599455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bede0-bb0c-4e99-9e16-a227146f11a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:46.602010 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bede0-bb0c-4e99-9e16-a227146f11a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:46.603830 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bede0-bb0c-4e99-9e16-a227146f11a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:46.604125 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bede0-bb0c-4e99-9e16-a227146f11a9 admin admin] PATH_INFO: `/v3/users/baf8f24ad16c47eeadd8b6e3a480836d` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:46.605651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770bede0-bb0c-4e99-9e16-a227146f11a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baf8f24ad16c47eeadd8b6e3a480836d)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:46.607763 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770bede0-bb0c-4e99-9e16-a227146f11a9 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:46.692889 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-770bede0-bb0c-4e99-9e16-a227146f11a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baf8f24ad16c47eeadd8b6e3a480836d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:46.710807 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-770bede0-bb0c-4e99-9e16-a227146f11a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baf8f24ad16c47eeadd8b6e3a480836d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:46.713043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-770bede0-bb0c-4e99-9e16-a227146f11a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baf8f24ad16c47eeadd8b6e3a480836d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:46.742390 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:46.745870 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-770bede0-bb0c-4e99-9e16-a227146f11a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baf8f24ad16c47eeadd8b6e3a480836d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:46.747654 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-770bede0-bb0c-4e99-9e16-a227146f11a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baf8f24ad16c47eeadd8b6e3a480836d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:46.773205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1093/2146] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:46 2022] DELETE /identity/v3/users/baf8f24ad16c47eeadd8b6e3a480836d => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:46.778290 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308a654-3b8a-4061-a26c-8be1e38e8069 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:46.778601 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308a654-3b8a-4061-a26c-8be1e38e8069 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:46.778909 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308a654-3b8a-4061-a26c-8be1e38e8069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:46.797007 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3308a654-3b8a-4061-a26c-8be1e38e8069 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:46.798284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3308a654-3b8a-4061-a26c-8be1e38e8069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:46.859876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1054/2147] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:46.873011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bede0-bb0c-4e99-9e16-a227146f11a9 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:46.874607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-770bede0-bb0c-4e99-9e16-a227146f11a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:46.951385 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bede0-bb0c-4e99-9e16-a227146f11a9 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:46.956016 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d532d-52c0-4ed6-8fc3-96558f611b95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:46.956758 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d532d-52c0-4ed6-8fc3-96558f611b95 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:46.957101 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d532d-52c0-4ed6-8fc3-96558f611b95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:46.957354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d532d-52c0-4ed6-8fc3-96558f611b95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:46.968661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d532d-52c0-4ed6-8fc3-96558f611b95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:46.970261 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d532d-52c0-4ed6-8fc3-96558f611b95 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:46.983631 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1094/2148] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:47.085345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308a654-3b8a-4061-a26c-8be1e38e8069 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:47.087377 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3308a654-3b8a-4061-a26c-8be1e38e8069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:47.101040 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308a654-3b8a-4061-a26c-8be1e38e8069 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:47.105784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2adbca4-c3c1-4004-9c25-3b9a4cb4a42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:47.106707 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adbca4-c3c1-4004-9c25-3b9a4cb4a42f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:47.106991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adbca4-c3c1-4004-9c25-3b9a4cb4a42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:47.107267 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adbca4-c3c1-4004-9c25-3b9a4cb4a42f admin admin] PATH_INFO: `/v3/users/35184312ee7b426db7a94d8d30e2c060` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:47.108507 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2adbca4-c3c1-4004-9c25-3b9a4cb4a42f admin admin] RBAC: Authorizing `identity:delete_user(user_id=35184312ee7b426db7a94d8d30e2c060)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:47.110956 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2adbca4-c3c1-4004-9c25-3b9a4cb4a42f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:47.162960 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:47.173311 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b2adbca4-c3c1-4004-9c25-3b9a4cb4a42f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35184312ee7b426db7a94d8d30e2c060'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:47.181971 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b2adbca4-c3c1-4004-9c25-3b9a4cb4a42f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35184312ee7b426db7a94d8d30e2c060'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:47.189328 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b2adbca4-c3c1-4004-9c25-3b9a4cb4a42f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35184312ee7b426db7a94d8d30e2c060'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:47.202594 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b2adbca4-c3c1-4004-9c25-3b9a4cb4a42f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35184312ee7b426db7a94d8d30e2c060'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:47.203352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b2adbca4-c3c1-4004-9c25-3b9a4cb4a42f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35184312ee7b426db7a94d8d30e2c060'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:47.209552 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1055/2149] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:47 2022] DELETE /identity/v3/users/35184312ee7b426db7a94d8d30e2c060 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:47.216065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f3581-6805-48ab-aef5-91b0a4a31a9b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:47.216549 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f3581-6805-48ab-aef5-91b0a4a31a9b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:47.217148 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f3581-6805-48ab-aef5-91b0a4a31a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:47.235124 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-300f3581-6805-48ab-aef5-91b0a4a31a9b None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:47.236723 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-300f3581-6805-48ab-aef5-91b0a4a31a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:47.295785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2adbca4-c3c1-4004-9c25-3b9a4cb4a42f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:47.297370 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b2adbca4-c3c1-4004-9c25-3b9a4cb4a42f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:47.333829 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1095/2150] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:47.347242 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f3581-6805-48ab-aef5-91b0a4a31a9b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:47.351803 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-300f3581-6805-48ab-aef5-91b0a4a31a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:47.450393 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2adbca4-c3c1-4004-9c25-3b9a4cb4a42f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:47.455625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eab142-622a-4a04-b630-42377f7f14c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:47.456739 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eab142-622a-4a04-b630-42377f7f14c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:47.457050 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eab142-622a-4a04-b630-42377f7f14c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:47.457432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eab142-622a-4a04-b630-42377f7f14c5 admin admin] PATH_INFO: `/v3/projects/3118a1c283ec4a1cb9a155bc849c15af` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:47.458628 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eab142-622a-4a04-b630-42377f7f14c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3118a1c283ec4a1cb9a155bc849c15af)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:47.460568 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eab142-622a-4a04-b630-42377f7f14c5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:47.474258 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f3581-6805-48ab-aef5-91b0a4a31a9b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:47.485801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68fc81-35e4-4ae9-8aef-e10d93d1ef04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:47.486491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68fc81-35e4-4ae9-8aef-e10d93d1ef04 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:47.486826 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68fc81-35e4-4ae9-8aef-e10d93d1ef04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:47.487109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68fc81-35e4-4ae9-8aef-e10d93d1ef04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:47.499532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68fc81-35e4-4ae9-8aef-e10d93d1ef04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:47.500975 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68fc81-35e4-4ae9-8aef-e10d93d1ef04 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:47.511005 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1096/2151] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:47.513389 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-30eab142-622a-4a04-b630-42377f7f14c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3118a1c283ec4a1cb9a155bc849c15af', 'request_id': 'req-30eab142-622a-4a04-b630-42377f7f14c5'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:47.519054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-30eab142-622a-4a04-b630-42377f7f14c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3118a1c283ec4a1cb9a155bc849c15af', 'request_id': 'req-30eab142-622a-4a04-b630-42377f7f14c5'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:47.525979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-30eab142-622a-4a04-b630-42377f7f14c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3118a1c283ec4a1cb9a155bc849c15af', 'request_id': 'req-30eab142-622a-4a04-b630-42377f7f14c5'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:47.545173 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-30eab142-622a-4a04-b630-42377f7f14c5 admin admin] The token cache is being invalidate because project 3118a1c283ec4a1cb9a155bc849c15af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:47.545540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-30eab142-622a-4a04-b630-42377f7f14c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:47.546275 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-30eab142-622a-4a04-b630-42377f7f14c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:47.548163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1056/2152] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:47 2022] DELETE /identity/v3/projects/3118a1c283ec4a1cb9a155bc849c15af => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:47.553158 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ab5fe-1861-4c0f-b6ad-4c15bed78db8 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:47.553449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ab5fe-1861-4c0f-b6ad-4c15bed78db8 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:47.553725 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ab5fe-1861-4c0f-b6ad-4c15bed78db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:47.574292 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-656ab5fe-1861-4c0f-b6ad-4c15bed78db8 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:47.575898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-656ab5fe-1861-4c0f-b6ad-4c15bed78db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:47.609975 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:47.663521 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1097/2153] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:47.672288 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682747b-2376-4792-91da-625d67e59bc2 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:47.672929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682747b-2376-4792-91da-625d67e59bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:47.673374 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682747b-2376-4792-91da-625d67e59bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:47.696622 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6682747b-2376-4792-91da-625d67e59bc2 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:47.697923 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6682747b-2376-4792-91da-625d67e59bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:47.706053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1057/2154] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:47.710327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ab5fe-1861-4c0f-b6ad-4c15bed78db8 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:47.736519 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ab5fe-1861-4c0f-b6ad-4c15bed78db8 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:47.755194 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e88008-09b4-4234-b2cc-057c9b887957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:47.755880 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e88008-09b4-4234-b2cc-057c9b887957 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:47.756532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e88008-09b4-4234-b2cc-057c9b887957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:47.756532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e88008-09b4-4234-b2cc-057c9b887957 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:47.757293 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e88008-09b4-4234-b2cc-057c9b887957 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:39:47.757606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e88008-09b4-4234-b2cc-057c9b887957 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:47.759299 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e88008-09b4-4234-b2cc-057c9b887957 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:47.770152 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682747b-2376-4792-91da-625d67e59bc2 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:47.771712 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6682747b-2376-4792-91da-625d67e59bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:47.776884 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1098/2155] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:39:47.781290 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e88008-09b4-4234-b2cc-057c9b887957 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:47.789092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682747b-2376-4792-91da-625d67e59bc2 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:47.800102 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c370c08-699f-48f2-8666-e2ee6939849c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:47.801367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c370c08-699f-48f2-8666-e2ee6939849c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:47.801792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c370c08-699f-48f2-8666-e2ee6939849c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:47.802211 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c370c08-699f-48f2-8666-e2ee6939849c admin admin] PATH_INFO: `/v3/users/fd5d1f7259034ab79dd3f601c136f415` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:47.804042 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c370c08-699f-48f2-8666-e2ee6939849c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd5d1f7259034ab79dd3f601c136f415)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:47.807614 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e88008-09b4-4234-b2cc-057c9b887957 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:47.808707 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c370c08-699f-48f2-8666-e2ee6939849c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:47.822092 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037c9f8-91a9-4b27-831b-f2069c19d378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:47.822934 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037c9f8-91a9-4b27-831b-f2069c19d378 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:47.823308 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037c9f8-91a9-4b27-831b-f2069c19d378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:47.823657 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037c9f8-91a9-4b27-831b-f2069c19d378 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:47.824424 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037c9f8-91a9-4b27-831b-f2069c19d378 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:47.826444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037c9f8-91a9-4b27-831b-f2069c19d378 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:47.874578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:47.890095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1099/2156] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:47 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:47.895093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037c9f8-91a9-4b27-831b-f2069c19d378 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:47.895991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7c370c08-699f-48f2-8666-e2ee6939849c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd5d1f7259034ab79dd3f601c136f415'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:47.898620 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3037c9f8-91a9-4b27-831b-f2069c19d378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:47.905059 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7c370c08-699f-48f2-8666-e2ee6939849c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd5d1f7259034ab79dd3f601c136f415'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:47.914477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7c370c08-699f-48f2-8666-e2ee6939849c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd5d1f7259034ab79dd3f601c136f415'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:47.931684 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7c370c08-699f-48f2-8666-e2ee6939849c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd5d1f7259034ab79dd3f601c136f415'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:47.932361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7c370c08-699f-48f2-8666-e2ee6939849c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd5d1f7259034ab79dd3f601c136f415'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:47.942616 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1058/2157] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:47 2022] DELETE /identity/v3/users/fd5d1f7259034ab79dd3f601c136f415 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:47.943542 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c370c08-699f-48f2-8666-e2ee6939849c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:47.945148 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7c370c08-699f-48f2-8666-e2ee6939849c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:48.017473 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037c9f8-91a9-4b27-831b-f2069c19d378 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:48.035497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c370c08-699f-48f2-8666-e2ee6939849c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:48.040456 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bf177-1472-4ec6-97ec-8836989c6689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:48.041140 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bf177-1472-4ec6-97ec-8836989c6689 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:48.041432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bf177-1472-4ec6-97ec-8836989c6689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:48.041716 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bf177-1472-4ec6-97ec-8836989c6689 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:48.042418 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48bf177-1472-4ec6-97ec-8836989c6689 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:48.044387 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48bf177-1472-4ec6-97ec-8836989c6689 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:48.071534 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1059/2158] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:47 2022] POST /identity/v3/users => generated 289 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:48.071534 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31104523-f852-429e-b59c-5eca0c5c50e0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:48.071534 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31104523-f852-429e-b59c-5eca0c5c50e0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:48.071534 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31104523-f852-429e-b59c-5eca0c5c50e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:48.088106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1d44f-79e2-4961-9697-9748955e43b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:48.088991 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1d44f-79e2-4961-9697-9748955e43b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:48.089410 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1d44f-79e2-4961-9697-9748955e43b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:48.089700 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1d44f-79e2-4961-9697-9748955e43b6 admin admin] PATH_INFO: `/v3/projects/0a9662d1e844401b839ee78652345f97` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:48.090945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1d44f-79e2-4961-9697-9748955e43b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a9662d1e844401b839ee78652345f97)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:48.092861 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1d44f-79e2-4961-9697-9748955e43b6 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:48.097414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-31104523-f852-429e-b59c-5eca0c5c50e0 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:48.098576 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-31104523-f852-429e-b59c-5eca0c5c50e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:48.106885 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1060/2159] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:48.110761 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31104523-f852-429e-b59c-5eca0c5c50e0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:48.123040 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31104523-f852-429e-b59c-5eca0c5c50e0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:48.129036 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01c8db-6465-4071-be2f-2c4b9826de66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:48.130194 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01c8db-6465-4071-be2f-2c4b9826de66 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:48.130501 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01c8db-6465-4071-be2f-2c4b9826de66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:48.130930 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01c8db-6465-4071-be2f-2c4b9826de66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:48.131672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e01c8db-6465-4071-be2f-2c4b9826de66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:48.133460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e01c8db-6465-4071-be2f-2c4b9826de66 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:48.144801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1061/2160] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:48.145665 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01c8db-6465-4071-be2f-2c4b9826de66 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:48.147356 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7e01c8db-6465-4071-be2f-2c4b9826de66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:48.148895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:48.152690 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-23f1d44f-79e2-4961-9697-9748955e43b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a9662d1e844401b839ee78652345f97', 'request_id': 'req-23f1d44f-79e2-4961-9697-9748955e43b6'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:48.160849 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-23f1d44f-79e2-4961-9697-9748955e43b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a9662d1e844401b839ee78652345f97', 'request_id': 'req-23f1d44f-79e2-4961-9697-9748955e43b6'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:48.168665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-23f1d44f-79e2-4961-9697-9748955e43b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a9662d1e844401b839ee78652345f97', 'request_id': 'req-23f1d44f-79e2-4961-9697-9748955e43b6'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:48.182952 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-23f1d44f-79e2-4961-9697-9748955e43b6 admin admin] The token cache is being invalidate because project 0a9662d1e844401b839ee78652345f97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:48.183436 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-23f1d44f-79e2-4961-9697-9748955e43b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:48.184028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-23f1d44f-79e2-4961-9697-9748955e43b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:48.185825 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1100/2161] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:47 2022] DELETE /identity/v3/projects/0a9662d1e844401b839ee78652345f97 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:48.186522 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1d44f-79e2-4961-9697-9748955e43b6 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:48.187853 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-23f1d44f-79e2-4961-9697-9748955e43b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:48.213619 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01c8db-6465-4071-be2f-2c4b9826de66 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:48.252172 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1d44f-79e2-4961-9697-9748955e43b6 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:48.254215 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:48.262141 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816c659-79c0-44c7-a065-659791067b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:48.263054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816c659-79c0-44c7-a065-659791067b33 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:48.263596 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816c659-79c0-44c7-a065-659791067b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:48.263919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816c659-79c0-44c7-a065-659791067b33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:48.264661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816c659-79c0-44c7-a065-659791067b33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:48.266535 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816c659-79c0-44c7-a065-659791067b33 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:48.276115 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38def3d5-140b-40e6-b164-ff1ff58f90ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:48.276884 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38def3d5-140b-40e6-b164-ff1ff58f90ef service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:48.277252 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38def3d5-140b-40e6-b164-ff1ff58f90ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:48.277612 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38def3d5-140b-40e6-b164-ff1ff58f90ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:48.279971 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-38def3d5-140b-40e6-b164-ff1ff58f90ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:48.280132 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1101/2162] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:48.280564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816c659-79c0-44c7-a065-659791067b33 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:48.282263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2816c659-79c0-44c7-a065-659791067b33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:48.294177 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38def3d5-140b-40e6-b164-ff1ff58f90ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:48.294683 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816c659-79c0-44c7-a065-659791067b33 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:48.295694 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38def3d5-140b-40e6-b164-ff1ff58f90ef service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:48.299332 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99ab81-b3fe-4234-86c1-f9fdaed2a817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:48.300182 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99ab81-b3fe-4234-86c1-f9fdaed2a817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:48.300774 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99ab81-b3fe-4234-86c1-f9fdaed2a817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:48.301059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99ab81-b3fe-4234-86c1-f9fdaed2a817 admin admin] PATH_INFO: `/v3/users/2302181bfe36431fb99b74c85dd03af5` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:48.302232 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99ab81-b3fe-4234-86c1-f9fdaed2a817 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2302181bfe36431fb99b74c85dd03af5)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:48.304203 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99ab81-b3fe-4234-86c1-f9fdaed2a817 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:48.305897 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1062/2163] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:48.306803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38def3d5-140b-40e6-b164-ff1ff58f90ef service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:48.316116 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38def3d5-140b-40e6-b164-ff1ff58f90ef service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:48.322101 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c01cf-a64b-4214-bbc3-33436d13a154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:48.322497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c01cf-a64b-4214-bbc3-33436d13a154 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:48.322891 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c01cf-a64b-4214-bbc3-33436d13a154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:48.323084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c01cf-a64b-4214-bbc3-33436d13a154 admin admin] PATH_INFO: `/v3/projects/2ab54c8c8832499087505dcec487852d/users/3c158f9c73ed4aa78d6d7916aa189607/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:48.341780 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570c01cf-a64b-4214-bbc3-33436d13a154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ab54c8c8832499087505dcec487852d, user_id=3c158f9c73ed4aa78d6d7916aa189607, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:48.345198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570c01cf-a64b-4214-bbc3-33436d13a154 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:48.361245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1063/2164] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:48 2022] PUT /identity/v3/projects/2ab54c8c8832499087505dcec487852d/users/3c158f9c73ed4aa78d6d7916aa189607/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:48.365250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0e99ab81-b3fe-4234-86c1-f9fdaed2a817 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2302181bfe36431fb99b74c85dd03af5'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:48.375872 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fdb3e9-8a9e-479d-a88d-ad44a7003921 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:48.376238 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fdb3e9-8a9e-479d-a88d-ad44a7003921 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:48.376549 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fdb3e9-8a9e-479d-a88d-ad44a7003921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:48.379533 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0e99ab81-b3fe-4234-86c1-f9fdaed2a817 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2302181bfe36431fb99b74c85dd03af5'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:48.380079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0e99ab81-b3fe-4234-86c1-f9fdaed2a817 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2302181bfe36431fb99b74c85dd03af5'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:48.396128 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0e99ab81-b3fe-4234-86c1-f9fdaed2a817 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2302181bfe36431fb99b74c85dd03af5'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:48.396843 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0e99ab81-b3fe-4234-86c1-f9fdaed2a817 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2302181bfe36431fb99b74c85dd03af5'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:48.407885 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e7fdb3e9-8a9e-479d-a88d-ad44a7003921 None None] MFA Rules not processed for user `3c158f9c73ed4aa78d6d7916aa189607`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:48.409384 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e7fdb3e9-8a9e-479d-a88d-ad44a7003921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:48.411632 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1102/2165] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:48 2022] DELETE /identity/v3/users/2302181bfe36431fb99b74c85dd03af5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:48.513396 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1064/2166] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:39:48.532730 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99ab81-b3fe-4234-86c1-f9fdaed2a817 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:48.534293 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0e99ab81-b3fe-4234-86c1-f9fdaed2a817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:48.590704 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:48.606318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99ab81-b3fe-4234-86c1-f9fdaed2a817 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:48.611191 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e637260-5e80-4f35-b03c-48b10dc5300d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8a4b0dd4e21414d91fe7bed478d15e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:48.611906 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e637260-5e80-4f35-b03c-48b10dc5300d service glance] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:48.612129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e637260-5e80-4f35-b03c-48b10dc5300d service glance] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:48.612403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e637260-5e80-4f35-b03c-48b10dc5300d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:48.623310 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e637260-5e80-4f35-b03c-48b10dc5300d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:48.625020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e637260-5e80-4f35-b03c-48b10dc5300d service glance] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:48.636478 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1103/2167] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:48.706012 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fdb3e9-8a9e-479d-a88d-ad44a7003921 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:48.707581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e7fdb3e9-8a9e-479d-a88d-ad44a7003921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:48.792747 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fdb3e9-8a9e-479d-a88d-ad44a7003921 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:48.798767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0217e8d-1f40-4e42-8f12-48434963a4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:48.799797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0217e8d-1f40-4e42-8f12-48434963a4c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:48.800124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0217e8d-1f40-4e42-8f12-48434963a4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:48.800439 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0217e8d-1f40-4e42-8f12-48434963a4c4 admin admin] PATH_INFO: `/v3/projects/0859e54615fa496781c30362138fcb5d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:48.801874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0217e8d-1f40-4e42-8f12-48434963a4c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0859e54615fa496781c30362138fcb5d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:48.803886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0217e8d-1f40-4e42-8f12-48434963a4c4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:48.828557 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e637260-5e80-4f35-b03c-48b10dc5300d service glance] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:48.830017 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6e637260-5e80-4f35-b03c-48b10dc5300d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:48.848200 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e637260-5e80-4f35-b03c-48b10dc5300d service glance] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:48.861252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364f3e1-6b77-45f3-a5c6-83913e5d37b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:48.862205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364f3e1-6b77-45f3-a5c6-83913e5d37b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:48.862493 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364f3e1-6b77-45f3-a5c6-83913e5d37b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:48.862803 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364f3e1-6b77-45f3-a5c6-83913e5d37b4 admin admin] PATH_INFO: `/v3/projects/426058ff5a0041a5bf371a0c9464e8a6` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:48.867965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:48.869389 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a0217e8d-1f40-4e42-8f12-48434963a4c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0859e54615fa496781c30362138fcb5d', 'request_id': 'req-a0217e8d-1f40-4e42-8f12-48434963a4c4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:48.873913 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364f3e1-6b77-45f3-a5c6-83913e5d37b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=426058ff5a0041a5bf371a0c9464e8a6)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:48.876881 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364f3e1-6b77-45f3-a5c6-83913e5d37b4 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:48.878027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a0217e8d-1f40-4e42-8f12-48434963a4c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0859e54615fa496781c30362138fcb5d', 'request_id': 'req-a0217e8d-1f40-4e42-8f12-48434963a4c4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:48.887440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a0217e8d-1f40-4e42-8f12-48434963a4c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0859e54615fa496781c30362138fcb5d', 'request_id': 'req-a0217e8d-1f40-4e42-8f12-48434963a4c4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:48.903883 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a0217e8d-1f40-4e42-8f12-48434963a4c4 admin admin] The token cache is being invalidate because project 0859e54615fa496781c30362138fcb5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:48.904177 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a0217e8d-1f40-4e42-8f12-48434963a4c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:48.904979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a0217e8d-1f40-4e42-8f12-48434963a4c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:48.907704 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1065/2168] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:48 2022] DELETE /identity/v3/projects/0859e54615fa496781c30362138fcb5d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:48.911631 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0217e8d-1f40-4e42-8f12-48434963a4c4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:48.913520 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a0217e8d-1f40-4e42-8f12-48434963a4c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:48.937983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a364f3e1-6b77-45f3-a5c6-83913e5d37b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '426058ff5a0041a5bf371a0c9464e8a6', 'request_id': 'req-a364f3e1-6b77-45f3-a5c6-83913e5d37b4'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:48.945119 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a364f3e1-6b77-45f3-a5c6-83913e5d37b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '426058ff5a0041a5bf371a0c9464e8a6', 'request_id': 'req-a364f3e1-6b77-45f3-a5c6-83913e5d37b4'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:48.952567 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a364f3e1-6b77-45f3-a5c6-83913e5d37b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '426058ff5a0041a5bf371a0c9464e8a6', 'request_id': 'req-a364f3e1-6b77-45f3-a5c6-83913e5d37b4'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:48.968711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a364f3e1-6b77-45f3-a5c6-83913e5d37b4 admin admin] The token cache is being invalidate because project 426058ff5a0041a5bf371a0c9464e8a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:48.969084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a364f3e1-6b77-45f3-a5c6-83913e5d37b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:48.969982 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a364f3e1-6b77-45f3-a5c6-83913e5d37b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:48.971905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1104/2169] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:48 2022] DELETE /identity/v3/projects/426058ff5a0041a5bf371a0c9464e8a6 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:48.978475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d45778-27f9-40ec-ad3c-6b6e21b51b1b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:48.979016 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d45778-27f9-40ec-ad3c-6b6e21b51b1b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:48.979564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d45778-27f9-40ec-ad3c-6b6e21b51b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:48.996672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0217e8d-1f40-4e42-8f12-48434963a4c4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:49.004291 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-61d45778-27f9-40ec-ad3c-6b6e21b51b1b None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:49.005742 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-61d45778-27f9-40ec-ad3c-6b6e21b51b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:49.073546 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222995cf-1135-4a55-b139-0105652e635c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:49.075289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222995cf-1135-4a55-b139-0105652e635c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:49.075581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222995cf-1135-4a55-b139-0105652e635c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:49.075888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222995cf-1135-4a55-b139-0105652e635c admin admin] PATH_INFO: `/v3/users/d35c5d8167014ed5ab1f5b8d3eb0b623` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:49.079033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222995cf-1135-4a55-b139-0105652e635c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d35c5d8167014ed5ab1f5b8d3eb0b623)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:49.081310 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222995cf-1135-4a55-b139-0105652e635c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:49.094415 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1105/2170] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:39:49.102961 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29784ad3-c025-4277-b6f2-d3837d5c6cc6 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:49.103405 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29784ad3-c025-4277-b6f2-d3837d5c6cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:49.103810 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29784ad3-c025-4277-b6f2-d3837d5c6cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:49.160888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-222995cf-1135-4a55-b139-0105652e635c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd35c5d8167014ed5ab1f5b8d3eb0b623'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:49.172739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-29784ad3-c025-4277-b6f2-d3837d5c6cc6 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:49.176025 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-29784ad3-c025-4277-b6f2-d3837d5c6cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:49.179308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-222995cf-1135-4a55-b139-0105652e635c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd35c5d8167014ed5ab1f5b8d3eb0b623'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:49.195210 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-222995cf-1135-4a55-b139-0105652e635c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd35c5d8167014ed5ab1f5b8d3eb0b623'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:49.200426 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1106/2171] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:49.202472 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29784ad3-c025-4277-b6f2-d3837d5c6cc6 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:49.204916 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-29784ad3-c025-4277-b6f2-d3837d5c6cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:49.223409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29784ad3-c025-4277-b6f2-d3837d5c6cc6 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:49.253378 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-222995cf-1135-4a55-b139-0105652e635c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd35c5d8167014ed5ab1f5b8d3eb0b623'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:49.255088 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-222995cf-1135-4a55-b139-0105652e635c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd35c5d8167014ed5ab1f5b8d3eb0b623'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:49.261015 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2189ad-f6bb-4035-9e10-b430cc3e6f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:49.262793 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2189ad-f6bb-4035-9e10-b430cc3e6f02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:49.263179 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2189ad-f6bb-4035-9e10-b430cc3e6f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:49.263407 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2189ad-f6bb-4035-9e10-b430cc3e6f02 admin admin] PATH_INFO: `/v3/users/36e6555dbc5e4c7d909d753798cf0f9b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:49.263689 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1066/2172] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:48 2022] DELETE /identity/v3/users/d35c5d8167014ed5ab1f5b8d3eb0b623 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:49.265108 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222995cf-1135-4a55-b139-0105652e635c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:49.265585 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2189ad-f6bb-4035-9e10-b430cc3e6f02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36e6555dbc5e4c7d909d753798cf0f9b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:49.266704 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-222995cf-1135-4a55-b139-0105652e635c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:49.268097 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2189ad-f6bb-4035-9e10-b430cc3e6f02 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:49.282734 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:49.349538 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ba2189ad-f6bb-4035-9e10-b430cc3e6f02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36e6555dbc5e4c7d909d753798cf0f9b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:49.366799 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ba2189ad-f6bb-4035-9e10-b430cc3e6f02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36e6555dbc5e4c7d909d753798cf0f9b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:49.368665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ba2189ad-f6bb-4035-9e10-b430cc3e6f02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36e6555dbc5e4c7d909d753798cf0f9b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:49.371722 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:49.388070 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222995cf-1135-4a55-b139-0105652e635c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:49.389725 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ba2189ad-f6bb-4035-9e10-b430cc3e6f02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36e6555dbc5e4c7d909d753798cf0f9b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:49.391411 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ba2189ad-f6bb-4035-9e10-b430cc3e6f02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36e6555dbc5e4c7d909d753798cf0f9b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:49.394161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8b356-c01e-4154-a7f8-9c1ecbc6e7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:49.394943 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8b356-c01e-4154-a7f8-9c1ecbc6e7d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:49.395324 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8b356-c01e-4154-a7f8-9c1ecbc6e7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:49.395656 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8b356-c01e-4154-a7f8-9c1ecbc6e7d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:49.396679 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8b356-c01e-4154-a7f8-9c1ecbc6e7d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:39:49.396810 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8b356-c01e-4154-a7f8-9c1ecbc6e7d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:49.398458 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8b356-c01e-4154-a7f8-9c1ecbc6e7d7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:49.409451 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1107/2173] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:49 2022] DELETE /identity/v3/users/36e6555dbc5e4c7d909d753798cf0f9b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:49.411035 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1067/2174] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:39:49.416145 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2189ad-f6bb-4035-9e10-b430cc3e6f02 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:49.418347 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd70d48-791e-4abd-a5d7-f058bd662095 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:49.418677 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd70d48-791e-4abd-a5d7-f058bd662095 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:49.420007 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd70d48-791e-4abd-a5d7-f058bd662095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:49.431802 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ba2189ad-f6bb-4035-9e10-b430cc3e6f02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:49.450789 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7dd70d48-791e-4abd-a5d7-f058bd662095 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:49.452037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7dd70d48-791e-4abd-a5d7-f058bd662095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:49.512162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2189ad-f6bb-4035-9e10-b430cc3e6f02 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:49.518753 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6849cc1-a3d9-42dc-a360-d0cc23f15de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:49.519528 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6849cc1-a3d9-42dc-a360-d0cc23f15de0 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:49.520266 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6849cc1-a3d9-42dc-a360-d0cc23f15de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:49.520535 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6849cc1-a3d9-42dc-a360-d0cc23f15de0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:49.521955 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6849cc1-a3d9-42dc-a360-d0cc23f15de0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:49.525389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6849cc1-a3d9-42dc-a360-d0cc23f15de0 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:49.583475 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1068/2175] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:49.592476 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd70d48-791e-4abd-a5d7-f058bd662095 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:49.595449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1108/2176] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:49 2022] POST /identity/v3/projects => generated 383 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:39:49.595600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7dd70d48-791e-4abd-a5d7-f058bd662095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:49.599324 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6849cc1-a3d9-42dc-a360-d0cc23f15de0 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:49.670738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6849cc1-a3d9-42dc-a360-d0cc23f15de0 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:49.671125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd70d48-791e-4abd-a5d7-f058bd662095 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:49.676823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d9fbf-737a-45e7-8f2a-0c0b7b65e2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:49.676823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d9fbf-737a-45e7-8f2a-0c0b7b65e2bc admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:49.676823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d9fbf-737a-45e7-8f2a-0c0b7b65e2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:49.677380 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d9fbf-737a-45e7-8f2a-0c0b7b65e2bc admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:49.678253 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d9fbf-737a-45e7-8f2a-0c0b7b65e2bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:49.680357 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d9fbf-737a-45e7-8f2a-0c0b7b65e2bc admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:49.704803 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1109/2177] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:49 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:39:49.705784 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d9fbf-737a-45e7-8f2a-0c0b7b65e2bc admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:49.709367 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-789d9fbf-737a-45e7-8f2a-0c0b7b65e2bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:49.729239 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d9fbf-737a-45e7-8f2a-0c0b7b65e2bc admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:49.734973 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967c5d4-b573-4ec9-952d-a8afe41a7da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:49.736077 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967c5d4-b573-4ec9-952d-a8afe41a7da4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:49.736414 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967c5d4-b573-4ec9-952d-a8afe41a7da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:49.737058 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967c5d4-b573-4ec9-952d-a8afe41a7da4 admin admin] PATH_INFO: `/v3/projects/7e17f4d9498842f882e2664c451499af` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:49.739460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9262b86-4fdd-4d8b-9266-c474fdf0c940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:49.739610 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a967c5d4-b573-4ec9-952d-a8afe41a7da4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e17f4d9498842f882e2664c451499af)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:49.740367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9262b86-4fdd-4d8b-9266-c474fdf0c940 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:49.740721 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9262b86-4fdd-4d8b-9266-c474fdf0c940 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:49.741087 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9262b86-4fdd-4d8b-9266-c474fdf0c940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:49.743801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a967c5d4-b573-4ec9-952d-a8afe41a7da4 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:49.750389 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:49.754302 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9262b86-4fdd-4d8b-9266-c474fdf0c940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:49.756956 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9262b86-4fdd-4d8b-9266-c474fdf0c940 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:49.770500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1069/2178] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:39:49.775753 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9262b86-4fdd-4d8b-9266-c474fdf0c940 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:49.793355 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9262b86-4fdd-4d8b-9266-c474fdf0c940 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:49.798326 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4ecfc-a8a7-4f89-9da8-1c3eb0930d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:49.799039 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4ecfc-a8a7-4f89-9da8-1c3eb0930d30 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:49.799367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4ecfc-a8a7-4f89-9da8-1c3eb0930d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:49.799599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4ecfc-a8a7-4f89-9da8-1c3eb0930d30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:49.800335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4ecfc-a8a7-4f89-9da8-1c3eb0930d30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:49.802368 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4ecfc-a8a7-4f89-9da8-1c3eb0930d30 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:49.810401 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:49.815951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a967c5d4-b573-4ec9-952d-a8afe41a7da4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e17f4d9498842f882e2664c451499af', 'request_id': 'req-a967c5d4-b573-4ec9-952d-a8afe41a7da4'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:49.816366 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1070/2179] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:49.820874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4ecfc-a8a7-4f89-9da8-1c3eb0930d30 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:49.833721 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a967c5d4-b573-4ec9-952d-a8afe41a7da4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e17f4d9498842f882e2664c451499af', 'request_id': 'req-a967c5d4-b573-4ec9-952d-a8afe41a7da4'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:49.834369 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4ecfc-a8a7-4f89-9da8-1c3eb0930d30 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:49.845965 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a967c5d4-b573-4ec9-952d-a8afe41a7da4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e17f4d9498842f882e2664c451499af', 'request_id': 'req-a967c5d4-b573-4ec9-952d-a8afe41a7da4'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:49.871072 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a967c5d4-b573-4ec9-952d-a8afe41a7da4 admin admin] The token cache is being invalidate because project 7e17f4d9498842f882e2664c451499af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:49.871298 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a967c5d4-b573-4ec9-952d-a8afe41a7da4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:49.872276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a967c5d4-b573-4ec9-952d-a8afe41a7da4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:49.874003 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1110/2180] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:49 2022] DELETE /identity/v3/projects/7e17f4d9498842f882e2664c451499af => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:49.880775 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea544b9-29ff-498c-a44d-00d0aca066aa None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:49.881080 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea544b9-29ff-498c-a44d-00d0aca066aa None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:49.881359 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea544b9-29ff-498c-a44d-00d0aca066aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:49.899823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-dea544b9-29ff-498c-a44d-00d0aca066aa None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:49.901050 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-dea544b9-29ff-498c-a44d-00d0aca066aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:49.924119 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a343736-33c3-4b53-8a77-634330a2fe47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:49.924690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a343736-33c3-4b53-8a77-634330a2fe47 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:49.924966 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a343736-33c3-4b53-8a77-634330a2fe47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:49.925241 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a343736-33c3-4b53-8a77-634330a2fe47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:49.926194 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a343736-33c3-4b53-8a77-634330a2fe47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:49.929055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a343736-33c3-4b53-8a77-634330a2fe47 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:49.953319 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1071/2181] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:39:49.957589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a343736-33c3-4b53-8a77-634330a2fe47 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:49.962249 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3a343736-33c3-4b53-8a77-634330a2fe47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:49.982919 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a343736-33c3-4b53-8a77-634330a2fe47 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:49.999580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1111/2182] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:49.999854 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94028e93-b0c6-404f-b102-c48a88185b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:49.999854 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94028e93-b0c6-404f-b102-c48a88185b03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:49.999854 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94028e93-b0c6-404f-b102-c48a88185b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:49.999854 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94028e93-b0c6-404f-b102-c48a88185b03 admin admin] PATH_INFO: `/v3/projects/00755c6e09e045cfb2b8f46d75b20a8a/users/b7c24486098e43c187e418cb33c600d7/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:50.001306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c8696-4fa3-4907-9dd6-ee4b36c7c1ea None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:50.002087 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c8696-4fa3-4907-9dd6-ee4b36c7c1ea None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:50.002383 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c8696-4fa3-4907-9dd6-ee4b36c7c1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:50.012580 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94028e93-b0c6-404f-b102-c48a88185b03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00755c6e09e045cfb2b8f46d75b20a8a, user_id=b7c24486098e43c187e418cb33c600d7, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:50.015345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94028e93-b0c6-404f-b102-c48a88185b03 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:50.022920 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-875c8696-4fa3-4907-9dd6-ee4b36c7c1ea None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:50.024757 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-875c8696-4fa3-4907-9dd6-ee4b36c7c1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:50.041849 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1072/2183] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:49 2022] PUT /identity/v3/projects/00755c6e09e045cfb2b8f46d75b20a8a/users/b7c24486098e43c187e418cb33c600d7/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:50.043652 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b250575-009b-49e0-ab77-870efb70ad25 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:50.043947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b250575-009b-49e0-ab77-870efb70ad25 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:50.044403 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b250575-009b-49e0-ab77-870efb70ad25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:50.080190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7b250575-009b-49e0-ab77-870efb70ad25 None None] MFA Rules not processed for user `b7c24486098e43c187e418cb33c600d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:50.081655 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7b250575-009b-49e0-ab77-870efb70ad25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:50.097002 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1112/2184] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:50.101243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c8696-4fa3-4907-9dd6-ee4b36c7c1ea None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:50.113137 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c8696-4fa3-4907-9dd6-ee4b36c7c1ea None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:50.121893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830a0a81-28e8-4462-a37e-538531fdb786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:50.122639 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a0a81-28e8-4462-a37e-538531fdb786 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:50.123042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a0a81-28e8-4462-a37e-538531fdb786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:50.123416 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a0a81-28e8-4462-a37e-538531fdb786 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:50.124258 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830a0a81-28e8-4462-a37e-538531fdb786 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:39:50.124668 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830a0a81-28e8-4462-a37e-538531fdb786 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:50.126548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830a0a81-28e8-4462-a37e-538531fdb786 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:50.142472 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1113/2185] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:39:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:39:50.146984 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830a0a81-28e8-4462-a37e-538531fdb786 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:50.158765 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830a0a81-28e8-4462-a37e-538531fdb786 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:50.167348 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce37522-18ba-46a3-93c5-2e83d6891db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:50.168095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce37522-18ba-46a3-93c5-2e83d6891db2 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:50.168443 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce37522-18ba-46a3-93c5-2e83d6891db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:50.168826 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce37522-18ba-46a3-93c5-2e83d6891db2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:50.169570 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce37522-18ba-46a3-93c5-2e83d6891db2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:50.171413 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1073/2186] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:50.172447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b250575-009b-49e0-ab77-870efb70ad25 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:50.173934 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7b250575-009b-49e0-ab77-870efb70ad25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:50.174775 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce37522-18ba-46a3-93c5-2e83d6891db2 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:50.189999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b250575-009b-49e0-ab77-870efb70ad25 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:50.194895 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703dbfee-f117-4c3d-9462-a58f752bea81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:50.196238 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703dbfee-f117-4c3d-9462-a58f752bea81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:50.196529 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703dbfee-f117-4c3d-9462-a58f752bea81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:50.196896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703dbfee-f117-4c3d-9462-a58f752bea81 admin admin] PATH_INFO: `/v3/projects/fd3b6eefba8a4f5aaeb5519969d9ce25` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:50.198271 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703dbfee-f117-4c3d-9462-a58f752bea81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd3b6eefba8a4f5aaeb5519969d9ce25)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:50.200514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703dbfee-f117-4c3d-9462-a58f752bea81 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:50.205482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1114/2187] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:50 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:50.206364 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce37522-18ba-46a3-93c5-2e83d6891db2 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:50.247898 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:50.253013 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:50.264164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-703dbfee-f117-4c3d-9462-a58f752bea81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd3b6eefba8a4f5aaeb5519969d9ce25', 'request_id': 'req-703dbfee-f117-4c3d-9462-a58f752bea81'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:50.272252 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-703dbfee-f117-4c3d-9462-a58f752bea81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd3b6eefba8a4f5aaeb5519969d9ce25', 'request_id': 'req-703dbfee-f117-4c3d-9462-a58f752bea81'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:50.284262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce37522-18ba-46a3-93c5-2e83d6891db2 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:50.286376 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-703dbfee-f117-4c3d-9462-a58f752bea81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd3b6eefba8a4f5aaeb5519969d9ce25', 'request_id': 'req-703dbfee-f117-4c3d-9462-a58f752bea81'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:50.303366 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-703dbfee-f117-4c3d-9462-a58f752bea81 admin admin] The token cache is being invalidate because project fd3b6eefba8a4f5aaeb5519969d9ce25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:50.303666 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-703dbfee-f117-4c3d-9462-a58f752bea81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:50.304330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-703dbfee-f117-4c3d-9462-a58f752bea81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:50.306741 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1074/2188] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:50 2022] DELETE /identity/v3/projects/fd3b6eefba8a4f5aaeb5519969d9ce25 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:50.307494 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703dbfee-f117-4c3d-9462-a58f752bea81 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:50.308822 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-703dbfee-f117-4c3d-9462-a58f752bea81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:50.366120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c34654-83e5-48c4-847a-387a227e21f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:50.366347 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c34654-83e5-48c4-847a-387a227e21f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:50.366631 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c34654-83e5-48c4-847a-387a227e21f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:50.366943 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c34654-83e5-48c4-847a-387a227e21f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:50.368165 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c34654-83e5-48c4-847a-387a227e21f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:50.370105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c34654-83e5-48c4-847a-387a227e21f2 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:50.392548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703dbfee-f117-4c3d-9462-a58f752bea81 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:50.397554 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1115/2189] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:50 2022] POST /identity/v3/projects => generated 383 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:50.398860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759625a4-cf25-4465-a48b-c3d34671121e None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:50.399674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759625a4-cf25-4465-a48b-c3d34671121e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:50.399958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759625a4-cf25-4465-a48b-c3d34671121e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:50.400412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501dc39-4cf2-462a-be74-ba6ddeee4dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:50.401819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501dc39-4cf2-462a-be74-ba6ddeee4dcf admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:50.402815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501dc39-4cf2-462a-be74-ba6ddeee4dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:50.403426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501dc39-4cf2-462a-be74-ba6ddeee4dcf admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:50.404389 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501dc39-4cf2-462a-be74-ba6ddeee4dcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:50.406814 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501dc39-4cf2-462a-be74-ba6ddeee4dcf admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:50.418885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-759625a4-cf25-4465-a48b-c3d34671121e None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:50.420522 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-759625a4-cf25-4465-a48b-c3d34671121e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:50.432912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1075/2190] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:50 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:50.433979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501dc39-4cf2-462a-be74-ba6ddeee4dcf admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:50.435851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8501dc39-4cf2-462a-be74-ba6ddeee4dcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:50.451896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501dc39-4cf2-462a-be74-ba6ddeee4dcf admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:50.495379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1116/2191] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:50.496122 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759625a4-cf25-4465-a48b-c3d34671121e None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:50.505322 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759625a4-cf25-4465-a48b-c3d34671121e None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:50.509982 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f9535-f9fa-4f4a-8dd8-8efb1709cc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:50.510660 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f9535-f9fa-4f4a-8dd8-8efb1709cc27 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:50.510954 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f9535-f9fa-4f4a-8dd8-8efb1709cc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:50.511233 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f9535-f9fa-4f4a-8dd8-8efb1709cc27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:50.511967 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f9535-f9fa-4f4a-8dd8-8efb1709cc27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:50.513567 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f9535-f9fa-4f4a-8dd8-8efb1709cc27 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:50.523816 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1117/2192] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:50.525550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12935a9c-9083-4c44-82cf-dd59eeb24a05 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:50.525948 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12935a9c-9083-4c44-82cf-dd59eeb24a05 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:50.526200 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffd0a4-332f-4f6b-971e-e2afd1c98302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:50.526386 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12935a9c-9083-4c44-82cf-dd59eeb24a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:50.527240 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffd0a4-332f-4f6b-971e-e2afd1c98302 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:50.528533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffd0a4-332f-4f6b-971e-e2afd1c98302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:50.528533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffd0a4-332f-4f6b-971e-e2afd1c98302 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:50.528773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffd0a4-332f-4f6b-971e-e2afd1c98302 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:50.530874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffd0a4-332f-4f6b-971e-e2afd1c98302 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:50.547406 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-12935a9c-9083-4c44-82cf-dd59eeb24a05 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:50.548647 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-12935a9c-9083-4c44-82cf-dd59eeb24a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:50.554411 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1076/2193] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:50 2022] POST /identity/v3/users => generated 298 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:50.554411 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffd0a4-332f-4f6b-971e-e2afd1c98302 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:50.557531 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1118/2194] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:50.558379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12935a9c-9083-4c44-82cf-dd59eeb24a05 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:50.563927 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffd0a4-332f-4f6b-971e-e2afd1c98302 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:50.568486 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a515fe30-8924-4514-aa60-6c54b0313fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:50.569890 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515fe30-8924-4514-aa60-6c54b0313fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:50.570080 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515fe30-8924-4514-aa60-6c54b0313fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:50.570376 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515fe30-8924-4514-aa60-6c54b0313fd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:50.571173 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a515fe30-8924-4514-aa60-6c54b0313fd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:50.572774 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a515fe30-8924-4514-aa60-6c54b0313fd3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:50.572920 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12935a9c-9083-4c44-82cf-dd59eeb24a05 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:50.577787 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe383592-a79a-4fe0-8278-757c447d6a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:50.579149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe383592-a79a-4fe0-8278-757c447d6a8f admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:50.579440 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe383592-a79a-4fe0-8278-757c447d6a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:50.579718 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe383592-a79a-4fe0-8278-757c447d6a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:50.580515 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe383592-a79a-4fe0-8278-757c447d6a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:50.582231 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe383592-a79a-4fe0-8278-757c447d6a8f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:50.584960 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1077/2195] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:50.589424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a515fe30-8924-4514-aa60-6c54b0313fd3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:50.605833 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1119/2196] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:50.605991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a515fe30-8924-4514-aa60-6c54b0313fd3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:50.606085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe383592-a79a-4fe0-8278-757c447d6a8f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:50.608725 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb1404-a8ed-4089-824e-09b4ba0339d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:50.609574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb1404-a8ed-4089-824e-09b4ba0339d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:50.609941 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb1404-a8ed-4089-824e-09b4ba0339d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:50.610341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb1404-a8ed-4089-824e-09b4ba0339d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:50.611373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fb1404-a8ed-4089-824e-09b4ba0339d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:39:50.611654 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fb1404-a8ed-4089-824e-09b4ba0339d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:50.613439 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fb1404-a8ed-4089-824e-09b4ba0339d4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:50.617867 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe383592-a79a-4fe0-8278-757c447d6a8f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:50.631074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1078/2197] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:39:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:39:50.631074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb1404-a8ed-4089-824e-09b4ba0339d4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:50.638605 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68594e64-c7ad-43bb-9808-60082a43a7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:50.639585 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68594e64-c7ad-43bb-9808-60082a43a7d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:50.639876 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68594e64-c7ad-43bb-9808-60082a43a7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:50.640164 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68594e64-c7ad-43bb-9808-60082a43a7d1 admin admin] PATH_INFO: `/v3/projects/60a94ab549b04c58b8341229fb716254/users/6f678d348f22411cb767f5afaeecfa50/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:50.658334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb1404-a8ed-4089-824e-09b4ba0339d4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:50.659339 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68594e64-c7ad-43bb-9808-60082a43a7d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60a94ab549b04c58b8341229fb716254, user_id=6f678d348f22411cb767f5afaeecfa50, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:50.662347 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68594e64-c7ad-43bb-9808-60082a43a7d1 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:50.663622 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce5729-ebf2-4b5d-892b-5ae8612675e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:50.665292 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce5729-ebf2-4b5d-892b-5ae8612675e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:50.665633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce5729-ebf2-4b5d-892b-5ae8612675e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:50.666248 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce5729-ebf2-4b5d-892b-5ae8612675e2 admin admin] PATH_INFO: `/v3/projects/7bb6bdd54d3d4a79a5300cd6d0bd2c1e/users/2a234857bf764bde848ef3a93b9b5c7c/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:50.677682 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1120/2198] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:50 2022] PUT /identity/v3/projects/60a94ab549b04c58b8341229fb716254/users/6f678d348f22411cb767f5afaeecfa50/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:50.678674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68594e64-c7ad-43bb-9808-60082a43a7d1 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:50.693615 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce5729-ebf2-4b5d-892b-5ae8612675e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bb6bdd54d3d4a79a5300cd6d0bd2c1e, user_id=2a234857bf764bde848ef3a93b9b5c7c, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:50.697121 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce5729-ebf2-4b5d-892b-5ae8612675e2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:50.705926 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:50.711991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=88,keystone:DELETE=4,keystone:INSERT=8 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:50.716637 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1079/2199] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:50 2022] PUT /identity/v3/projects/7bb6bdd54d3d4a79a5300cd6d0bd2c1e/users/2a234857bf764bde848ef3a93b9b5c7c/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:50.718417 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea36a0-e1fb-4231-bbe4-23958712a206 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:50.718815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea36a0-e1fb-4231-bbe4-23958712a206 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:50.719125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea36a0-e1fb-4231-bbe4-23958712a206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:50.750334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-31ea36a0-e1fb-4231-bbe4-23958712a206 None None] MFA Rules not processed for user `6f678d348f22411cb767f5afaeecfa50`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:50.752985 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-31ea36a0-e1fb-4231-bbe4-23958712a206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:50.764548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68594e64-c7ad-43bb-9808-60082a43a7d1 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:50.831270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1080/2200] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:50.834675 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e70db-471b-4ac2-94fb-dd291e7b2ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:50.839192 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e70db-471b-4ac2-94fb-dd291e7b2ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:50.839507 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea36a0-e1fb-4231-bbe4-23958712a206 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:50.841287 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e70db-471b-4ac2-94fb-dd291e7b2ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:50.841683 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e70db-471b-4ac2-94fb-dd291e7b2ac4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:50.842332 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e70db-471b-4ac2-94fb-dd291e7b2ac4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:50.844396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e70db-471b-4ac2-94fb-dd291e7b2ac4 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:50.853240 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea36a0-e1fb-4231-bbe4-23958712a206 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:50.858972 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc10fb-587b-4f16-a401-87f26bcee660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:50.859817 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc10fb-587b-4f16-a401-87f26bcee660 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:50.860456 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc10fb-587b-4f16-a401-87f26bcee660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:50.860690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc10fb-587b-4f16-a401-87f26bcee660 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:50.861642 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc10fb-587b-4f16-a401-87f26bcee660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:50.863660 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc10fb-587b-4f16-a401-87f26bcee660 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:50.875308 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1121/2201] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:50 2022] POST /identity/v3/projects => generated 377 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:50.876242 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e70db-471b-4ac2-94fb-dd291e7b2ac4 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:50.877082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1081/2202] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:50.877936 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e36e70db-471b-4ac2-94fb-dd291e7b2ac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:50.880262 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc10fb-587b-4f16-a401-87f26bcee660 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:50.892262 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc10fb-587b-4f16-a401-87f26bcee660 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:50.951172 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e70db-471b-4ac2-94fb-dd291e7b2ac4 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:50.955666 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d215a3f-0f28-40d7-9f0d-429f8172bbc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:50.956335 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d215a3f-0f28-40d7-9f0d-429f8172bbc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:50.956630 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d215a3f-0f28-40d7-9f0d-429f8172bbc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:50.956913 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d215a3f-0f28-40d7-9f0d-429f8172bbc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:50.964265 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3040ec6-4c2f-4344-b363-4e95d18e4313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:50.964871 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3040ec6-4c2f-4344-b363-4e95d18e4313 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:50.965166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3040ec6-4c2f-4344-b363-4e95d18e4313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:50.965451 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3040ec6-4c2f-4344-b363-4e95d18e4313 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:50.966918 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3040ec6-4c2f-4344-b363-4e95d18e4313 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:50.969436 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3040ec6-4c2f-4344-b363-4e95d18e4313 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:50.994555 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1082/2203] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:50 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:50.995410 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3040ec6-4c2f-4344-b363-4e95d18e4313 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:51.006315 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3040ec6-4c2f-4344-b363-4e95d18e4313 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:51.011443 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42799d34-6b68-4148-afeb-ae381233314f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:51.012258 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42799d34-6b68-4148-afeb-ae381233314f admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:51.012560 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42799d34-6b68-4148-afeb-ae381233314f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:51.012829 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42799d34-6b68-4148-afeb-ae381233314f admin admin] PATH_INFO: `/v3/projects/7bb6bdd54d3d4a79a5300cd6d0bd2c1e/users/2a234857bf764bde848ef3a93b9b5c7c/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:51.014873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42799d34-6b68-4148-afeb-ae381233314f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bb6bdd54d3d4a79a5300cd6d0bd2c1e, user_id=2a234857bf764bde848ef3a93b9b5c7c, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:51.017555 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42799d34-6b68-4148-afeb-ae381233314f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:51.040208 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1083/2204] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:50 2022] PUT /identity/v3/projects/7bb6bdd54d3d4a79a5300cd6d0bd2c1e/users/2a234857bf764bde848ef3a93b9b5c7c/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:51.040208 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42799d34-6b68-4148-afeb-ae381233314f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:51.040567 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d215a3f-0f28-40d7-9f0d-429f8172bbc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:51.040991 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d215a3f-0f28-40d7-9f0d-429f8172bbc8 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:51.051165 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1122/2205] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:51.052516 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444aa3a-dd3a-4878-b7e8-8ca93b612185 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:51.053365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444aa3a-dd3a-4878-b7e8-8ca93b612185 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:51.053365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444aa3a-dd3a-4878-b7e8-8ca93b612185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:51.061663 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:51.084983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-0444aa3a-dd3a-4878-b7e8-8ca93b612185 None None] MFA Rules not processed for user `2a234857bf764bde848ef3a93b9b5c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:51.087190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0444aa3a-dd3a-4878-b7e8-8ca93b612185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:51.098840 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:51.105393 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42799d34-6b68-4148-afeb-ae381233314f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:51.111627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c552a32-5a29-479b-8a2a-c94dd9021f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:51.112458 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c552a32-5a29-479b-8a2a-c94dd9021f8f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:51.112979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c552a32-5a29-479b-8a2a-c94dd9021f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:51.113173 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c552a32-5a29-479b-8a2a-c94dd9021f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:51.113903 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c552a32-5a29-479b-8a2a-c94dd9021f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:51.115583 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c552a32-5a29-479b-8a2a-c94dd9021f8f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:51.128315 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1084/2206] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:51.135005 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c552a32-5a29-479b-8a2a-c94dd9021f8f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:51.146004 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c552a32-5a29-479b-8a2a-c94dd9021f8f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:51.151765 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40331b5d-9b1b-4e5b-92d1-e5ae3cc02934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:51.152524 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40331b5d-9b1b-4e5b-92d1-e5ae3cc02934 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:51.152868 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40331b5d-9b1b-4e5b-92d1-e5ae3cc02934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:51.153147 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40331b5d-9b1b-4e5b-92d1-e5ae3cc02934 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:51.153904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40331b5d-9b1b-4e5b-92d1-e5ae3cc02934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:51.155655 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40331b5d-9b1b-4e5b-92d1-e5ae3cc02934 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:51.168030 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1085/2207] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:51.172092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40331b5d-9b1b-4e5b-92d1-e5ae3cc02934 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:51.178341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1123/2208] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:51.183980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40331b5d-9b1b-4e5b-92d1-e5ae3cc02934 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:51.188915 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c7014-1354-4f57-9214-d8e132bab825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:51.190132 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c7014-1354-4f57-9214-d8e132bab825 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:51.190419 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c7014-1354-4f57-9214-d8e132bab825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:51.190888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c7014-1354-4f57-9214-d8e132bab825 admin admin] PATH_INFO: `/v3/projects/88361a67b4694d48a98f01350a1c35eb/users/ab1506823ed5423d8056af2818140672/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:51.191179 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444aa3a-dd3a-4878-b7e8-8ca93b612185 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:51.192799 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0444aa3a-dd3a-4878-b7e8-8ca93b612185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:51.212163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c7014-1354-4f57-9214-d8e132bab825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88361a67b4694d48a98f01350a1c35eb, user_id=ab1506823ed5423d8056af2818140672, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:51.215164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c7014-1354-4f57-9214-d8e132bab825 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:51.241337 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1086/2209] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:51 2022] PUT /identity/v3/projects/88361a67b4694d48a98f01350a1c35eb/users/ab1506823ed5423d8056af2818140672/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:51.245578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631dc0b-832f-4cb6-9d16-97dc0d941585 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:51.246056 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631dc0b-832f-4cb6-9d16-97dc0d941585 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:51.246349 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631dc0b-832f-4cb6-9d16-97dc0d941585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:51.271170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444aa3a-dd3a-4878-b7e8-8ca93b612185 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:51.277977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7631dc0b-832f-4cb6-9d16-97dc0d941585 None None] MFA Rules not processed for user `ab1506823ed5423d8056af2818140672`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:51.279391 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7631dc0b-832f-4cb6-9d16-97dc0d941585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:51.337358 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c1d86f-0f0e-4c3c-a8fa-1b53385d4a49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:51.338050 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1d86f-0f0e-4c3c-a8fa-1b53385d4a49 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:51.338299 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1d86f-0f0e-4c3c-a8fa-1b53385d4a49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:51.338579 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1d86f-0f0e-4c3c-a8fa-1b53385d4a49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:51.378878 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1087/2210] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:51.380397 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631dc0b-832f-4cb6-9d16-97dc0d941585 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:51.429820 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:51.433922 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c1d86f-0f0e-4c3c-a8fa-1b53385d4a49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:51.436350 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c1d86f-0f0e-4c3c-a8fa-1b53385d4a49 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:51.455720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1124/2211] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:39:51.507578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631dc0b-832f-4cb6-9d16-97dc0d941585 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:51.512789 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae330ef-201d-4a83-b41e-0381f00a1df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:51.513677 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae330ef-201d-4a83-b41e-0381f00a1df8 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:51.514108 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae330ef-201d-4a83-b41e-0381f00a1df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:51.514480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae330ef-201d-4a83-b41e-0381f00a1df8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:51.515427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae330ef-201d-4a83-b41e-0381f00a1df8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:51.517605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae330ef-201d-4a83-b41e-0381f00a1df8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:51.541971 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c1d86f-0f0e-4c3c-a8fa-1b53385d4a49 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:51.551065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c1d86f-0f0e-4c3c-a8fa-1b53385d4a49 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:51.552638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1088/2212] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:51 2022] POST /identity/v3/projects => generated 375 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:51.557274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae330ef-201d-4a83-b41e-0381f00a1df8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:51.594447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:51.613764 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636aa46-541e-494c-8aa8-7d111fc80ccf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:51.614438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636aa46-541e-494c-8aa8-7d111fc80ccf service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:51.614754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636aa46-541e-494c-8aa8-7d111fc80ccf service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:51.615042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636aa46-541e-494c-8aa8-7d111fc80ccf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:51.616936 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1636aa46-541e-494c-8aa8-7d111fc80ccf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:51.654137 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae330ef-201d-4a83-b41e-0381f00a1df8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:51.661682 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c5292-0f20-4f0f-ab54-4754008ee807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:51.662906 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c5292-0f20-4f0f-ab54-4754008ee807 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:51.663233 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c5292-0f20-4f0f-ab54-4754008ee807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:51.663484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c5292-0f20-4f0f-ab54-4754008ee807 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:51.665286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c5292-0f20-4f0f-ab54-4754008ee807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:51.667636 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c5292-0f20-4f0f-ab54-4754008ee807 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:51.693436 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1089/2213] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:51 2022] POST /identity/v3/users => generated 288 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:39:51.697592 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c5292-0f20-4f0f-ab54-4754008ee807 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:51.704879 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636aa46-541e-494c-8aa8-7d111fc80ccf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:51.706327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636aa46-541e-494c-8aa8-7d111fc80ccf service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:51.707768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c5292-0f20-4f0f-ab54-4754008ee807 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:51.712517 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24643d20-ef42-44e7-b50a-bbc37f454e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:51.713257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24643d20-ef42-44e7-b50a-bbc37f454e76 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:51.713587 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24643d20-ef42-44e7-b50a-bbc37f454e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:51.713870 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24643d20-ef42-44e7-b50a-bbc37f454e76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:51.714673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24643d20-ef42-44e7-b50a-bbc37f454e76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:51.715819 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1125/2214] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:39:51.719138 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24643d20-ef42-44e7-b50a-bbc37f454e76 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:51.739166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1090/2215] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:51.744551 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636aa46-541e-494c-8aa8-7d111fc80ccf service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:51.760266 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636aa46-541e-494c-8aa8-7d111fc80ccf service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:51.764897 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ee2c2-7631-4e80-a748-5cb792f439ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:51.765565 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ee2c2-7631-4e80-a748-5cb792f439ba admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:51.765856 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ee2c2-7631-4e80-a748-5cb792f439ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:51.766134 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ee2c2-7631-4e80-a748-5cb792f439ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:51.766900 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ee2c2-7631-4e80-a748-5cb792f439ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:51.768712 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ee2c2-7631-4e80-a748-5cb792f439ba admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:51.793507 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1126/2216] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:51.794693 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24643d20-ef42-44e7-b50a-bbc37f454e76 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:51.806635 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24643d20-ef42-44e7-b50a-bbc37f454e76 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:51.813272 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff5bb4-9910-4001-be69-aec1e249cdbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:51.814636 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff5bb4-9910-4001-be69-aec1e249cdbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:51.814950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff5bb4-9910-4001-be69-aec1e249cdbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:51.815229 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff5bb4-9910-4001-be69-aec1e249cdbd admin admin] PATH_INFO: `/v3/projects/5b4fda417c7f4472b7d919554e1a153e/users/c34f02289ccf43488994f98d2ab50048/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:51.834844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff5bb4-9910-4001-be69-aec1e249cdbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b4fda417c7f4472b7d919554e1a153e, user_id=c34f02289ccf43488994f98d2ab50048, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:51.837970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff5bb4-9910-4001-be69-aec1e249cdbd admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:51.856291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1091/2217] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:51 2022] PUT /identity/v3/projects/5b4fda417c7f4472b7d919554e1a153e/users/c34f02289ccf43488994f98d2ab50048/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:51.866024 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa3dd0-6437-42be-8457-de90a894a0b4 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:51.866821 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa3dd0-6437-42be-8457-de90a894a0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:51.866821 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa3dd0-6437-42be-8457-de90a894a0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:51.901331 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-17fa3dd0-6437-42be-8457-de90a894a0b4 None None] MFA Rules not processed for user `c34f02289ccf43488994f98d2ab50048`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:51.902606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-17fa3dd0-6437-42be-8457-de90a894a0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:51.982463 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1127/2218] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:51.987400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff5bb4-9910-4001-be69-aec1e249cdbd admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:52.008322 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fa3dd0-6437-42be-8457-de90a894a0b4 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:52.044313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:52.058287 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff5bb4-9910-4001-be69-aec1e249cdbd admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:52.063556 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7255ff4-5dfb-48f7-ac95-5a96dd1efd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:52.064581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7255ff4-5dfb-48f7-ac95-5a96dd1efd34 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:52.064935 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7255ff4-5dfb-48f7-ac95-5a96dd1efd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:52.065439 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7255ff4-5dfb-48f7-ac95-5a96dd1efd34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:52.066352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7255ff4-5dfb-48f7-ac95-5a96dd1efd34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:52.068561 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7255ff4-5dfb-48f7-ac95-5a96dd1efd34 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:52.097129 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1092/2219] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:51 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:39:52.102970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7255ff4-5dfb-48f7-ac95-5a96dd1efd34 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:52.103994 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fa3dd0-6437-42be-8457-de90a894a0b4 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:52.182948 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc02a9-495b-49db-b1ce-bf2fc20861e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:52.183824 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc02a9-495b-49db-b1ce-bf2fc20861e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:52.184183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc02a9-495b-49db-b1ce-bf2fc20861e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:52.184499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc02a9-495b-49db-b1ce-bf2fc20861e8 admin admin] PATH_INFO: `/v3/users/b7c24486098e43c187e418cb33c600d7` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:52.185701 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc02a9-495b-49db-b1ce-bf2fc20861e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7c24486098e43c187e418cb33c600d7)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:52.187681 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc02a9-495b-49db-b1ce-bf2fc20861e8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:52.188663 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7255ff4-5dfb-48f7-ac95-5a96dd1efd34 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:52.194962 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b84377-6e5a-43b0-83a4-6c5bfcb2c89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:52.197120 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b84377-6e5a-43b0-83a4-6c5bfcb2c89c admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:52.197416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b84377-6e5a-43b0-83a4-6c5bfcb2c89c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:52.197700 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b84377-6e5a-43b0-83a4-6c5bfcb2c89c admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:52.198404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b84377-6e5a-43b0-83a4-6c5bfcb2c89c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:52.200379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b84377-6e5a-43b0-83a4-6c5bfcb2c89c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:52.221668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:52.228431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1093/2220] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:52 2022] POST /identity/v3/users => generated 295 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:39:52.234477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b84377-6e5a-43b0-83a4-6c5bfcb2c89c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:52.250897 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f1cc02a9-495b-49db-b1ce-bf2fc20861e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7c24486098e43c187e418cb33c600d7'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:52.252050 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b84377-6e5a-43b0-83a4-6c5bfcb2c89c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:52.257530 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321db736-2c29-41d8-b281-b81b2bc60e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:52.258345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321db736-2c29-41d8-b281-b81b2bc60e31 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:52.258708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321db736-2c29-41d8-b281-b81b2bc60e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:52.258974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321db736-2c29-41d8-b281-b81b2bc60e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:52.259710 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321db736-2c29-41d8-b281-b81b2bc60e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:52.259906 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f1cc02a9-495b-49db-b1ce-bf2fc20861e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7c24486098e43c187e418cb33c600d7'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:52.260556 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f1cc02a9-495b-49db-b1ce-bf2fc20861e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7c24486098e43c187e418cb33c600d7'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:52.261325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321db736-2c29-41d8-b281-b81b2bc60e31 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:52.276347 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f1cc02a9-495b-49db-b1ce-bf2fc20861e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7c24486098e43c187e418cb33c600d7'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:52.277135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f1cc02a9-495b-49db-b1ce-bf2fc20861e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7c24486098e43c187e418cb33c600d7'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:52.280823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1094/2221] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:52.281913 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321db736-2c29-41d8-b281-b81b2bc60e31 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:52.284041 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-321db736-2c29-41d8-b281-b81b2bc60e31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:52.291470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1128/2222] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:52 2022] DELETE /identity/v3/users/b7c24486098e43c187e418cb33c600d7 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:52.296011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0b9ae-0d24-4402-9379-ec244ceede27 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:52.296329 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0b9ae-0d24-4402-9379-ec244ceede27 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:52.296655 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0b9ae-0d24-4402-9379-ec244ceede27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:52.299555 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321db736-2c29-41d8-b281-b81b2bc60e31 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:52.316314 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-8ce0b9ae-0d24-4402-9379-ec244ceede27 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:52.317544 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8ce0b9ae-0d24-4402-9379-ec244ceede27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:52.359124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2fa2f-16c2-4228-9d17-8d10df5a3564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:52.360092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2fa2f-16c2-4228-9d17-8d10df5a3564 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:52.360381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2fa2f-16c2-4228-9d17-8d10df5a3564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:52.360664 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2fa2f-16c2-4228-9d17-8d10df5a3564 admin admin] PATH_INFO: `/v3/projects/338b138d409f4447b823202c3c028ae1/users/86123a2a28cc43eca7d8c20fa1078857/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:52.369208 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:52.376012 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2fa2f-16c2-4228-9d17-8d10df5a3564 admin admin] RBAC: Authorizing `identity:create_grant(project_id=338b138d409f4447b823202c3c028ae1, user_id=86123a2a28cc43eca7d8c20fa1078857, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:52.378846 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2fa2f-16c2-4228-9d17-8d10df5a3564 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:52.384782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1129/2223] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:39:52.393240 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1095/2224] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:52 2022] PUT /identity/v3/projects/338b138d409f4447b823202c3c028ae1/users/86123a2a28cc43eca7d8c20fa1078857/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:39:52.395836 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce0b9ae-0d24-4402-9379-ec244ceede27 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:52.396904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2fa2f-16c2-4228-9d17-8d10df5a3564 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:52.397369 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8ce0b9ae-0d24-4402-9379-ec244ceede27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:52.464764 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2fa2f-16c2-4228-9d17-8d10df5a3564 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:52.466109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce0b9ae-0d24-4402-9379-ec244ceede27 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:52.470092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba9143-4486-45ca-8fdf-c56ebc4baed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:52.471180 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba9143-4486-45ca-8fdf-c56ebc4baed7 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:52.471573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba9143-4486-45ca-8fdf-c56ebc4baed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:52.472224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba9143-4486-45ca-8fdf-c56ebc4baed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:52.473221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba9143-4486-45ca-8fdf-c56ebc4baed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:52.475045 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba9143-4486-45ca-8fdf-c56ebc4baed7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:52.489343 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1096/2225] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:39:52.494915 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba9143-4486-45ca-8fdf-c56ebc4baed7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:52.506062 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba9143-4486-45ca-8fdf-c56ebc4baed7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:52.512027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ead0e-87b1-46ad-bafd-467583bc5ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:52.513185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ead0e-87b1-46ad-bafd-467583bc5ab0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:52.513374 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ead0e-87b1-46ad-bafd-467583bc5ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:52.513706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ead0e-87b1-46ad-bafd-467583bc5ab0 admin admin] PATH_INFO: `/v3/projects/338b138d409f4447b823202c3c028ae1/users/86123a2a28cc43eca7d8c20fa1078857/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:52.516200 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ead0e-87b1-46ad-bafd-467583bc5ab0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=338b138d409f4447b823202c3c028ae1, user_id=86123a2a28cc43eca7d8c20fa1078857, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:52.518871 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ead0e-87b1-46ad-bafd-467583bc5ab0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:52.530182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1097/2226] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:52 2022] PUT /identity/v3/projects/338b138d409f4447b823202c3c028ae1/users/86123a2a28cc43eca7d8c20fa1078857/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:52.539033 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20de007-01b8-4973-9d02-0c361ee64e51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:52.541730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522adb6f-ed7c-468d-8101-63b9a3262e67 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:52.542098 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522adb6f-ed7c-468d-8101-63b9a3262e67 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:52.542227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20de007-01b8-4973-9d02-0c361ee64e51 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:52.542385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522adb6f-ed7c-468d-8101-63b9a3262e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:52.542492 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20de007-01b8-4973-9d02-0c361ee64e51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:52.542753 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20de007-01b8-4973-9d02-0c361ee64e51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:52.553073 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20de007-01b8-4973-9d02-0c361ee64e51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:52.554689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20de007-01b8-4973-9d02-0c361ee64e51 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:52.566029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1130/2227] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:52.570759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-522adb6f-ed7c-468d-8101-63b9a3262e67 None None] MFA Rules not processed for user `86123a2a28cc43eca7d8c20fa1078857`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:52.571997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-522adb6f-ed7c-468d-8101-63b9a3262e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:52.622855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:52.650951 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1098/2228] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:52 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:52.662475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20de007-01b8-4973-9d02-0c361ee64e51 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:52.663913 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a20de007-01b8-4973-9d02-0c361ee64e51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:52.745666 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20de007-01b8-4973-9d02-0c361ee64e51 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:52.802598 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6638f1-bbdb-4887-b10d-2186b01ad933 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:52.803287 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6638f1-bbdb-4887-b10d-2186b01ad933 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:52.803563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6638f1-bbdb-4887-b10d-2186b01ad933 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:52.803874 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6638f1-bbdb-4887-b10d-2186b01ad933 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:52.805813 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ca6638f1-bbdb-4887-b10d-2186b01ad933 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:52.836900 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:52.877840 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6638f1-bbdb-4887-b10d-2186b01ad933 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:52.879381 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6638f1-bbdb-4887-b10d-2186b01ad933 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:52.889958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1131/2229] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:39:52.959634 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522adb6f-ed7c-468d-8101-63b9a3262e67 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:52.961978 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-522adb6f-ed7c-468d-8101-63b9a3262e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:53.067409 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522adb6f-ed7c-468d-8101-63b9a3262e67 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:53.074771 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bc08f-86db-4fe1-bd0f-4315db7aad31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:53.075977 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bc08f-86db-4fe1-bd0f-4315db7aad31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:53.076417 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bc08f-86db-4fe1-bd0f-4315db7aad31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:53.076703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bc08f-86db-4fe1-bd0f-4315db7aad31 admin admin] PATH_INFO: `/v3/projects/00755c6e09e045cfb2b8f46d75b20a8a` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:53.078018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846bc08f-86db-4fe1-bd0f-4315db7aad31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00755c6e09e045cfb2b8f46d75b20a8a)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:53.080025 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846bc08f-86db-4fe1-bd0f-4315db7aad31 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:53.139030 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-846bc08f-86db-4fe1-bd0f-4315db7aad31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00755c6e09e045cfb2b8f46d75b20a8a', 'request_id': 'req-846bc08f-86db-4fe1-bd0f-4315db7aad31'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:53.144939 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-846bc08f-86db-4fe1-bd0f-4315db7aad31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00755c6e09e045cfb2b8f46d75b20a8a', 'request_id': 'req-846bc08f-86db-4fe1-bd0f-4315db7aad31'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:53.151975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-846bc08f-86db-4fe1-bd0f-4315db7aad31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00755c6e09e045cfb2b8f46d75b20a8a', 'request_id': 'req-846bc08f-86db-4fe1-bd0f-4315db7aad31'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:53.166823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-846bc08f-86db-4fe1-bd0f-4315db7aad31 admin admin] The token cache is being invalidate because project 00755c6e09e045cfb2b8f46d75b20a8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:53.167300 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-846bc08f-86db-4fe1-bd0f-4315db7aad31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:53.167980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-846bc08f-86db-4fe1-bd0f-4315db7aad31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:53.177363 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1099/2230] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:52 2022] DELETE /identity/v3/projects/00755c6e09e045cfb2b8f46d75b20a8a => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:53.184218 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6638f1-bbdb-4887-b10d-2186b01ad933 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:53.198586 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6638f1-bbdb-4887-b10d-2186b01ad933 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:53.271080 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970013b-46e2-42cf-a966-ea6b99c4fe4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:53.272185 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970013b-46e2-42cf-a966-ea6b99c4fe4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:53.272602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970013b-46e2-42cf-a966-ea6b99c4fe4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:53.272946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970013b-46e2-42cf-a966-ea6b99c4fe4d admin admin] PATH_INFO: `/v3/users/2a234857bf764bde848ef3a93b9b5c7c` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:53.274874 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970013b-46e2-42cf-a966-ea6b99c4fe4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a234857bf764bde848ef3a93b9b5c7c)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:53.276999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970013b-46e2-42cf-a966-ea6b99c4fe4d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:53.339571 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a970013b-46e2-42cf-a966-ea6b99c4fe4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a234857bf764bde848ef3a93b9b5c7c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:53.348087 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a970013b-46e2-42cf-a966-ea6b99c4fe4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a234857bf764bde848ef3a93b9b5c7c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:53.348717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a970013b-46e2-42cf-a966-ea6b99c4fe4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a234857bf764bde848ef3a93b9b5c7c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:53.363973 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a970013b-46e2-42cf-a966-ea6b99c4fe4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a234857bf764bde848ef3a93b9b5c7c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:53.364682 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a970013b-46e2-42cf-a966-ea6b99c4fe4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a234857bf764bde848ef3a93b9b5c7c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:53.377880 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1132/2231] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:39:53 2022] DELETE /identity/v3/users/2a234857bf764bde848ef3a93b9b5c7c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:53.933593 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bc08f-86db-4fe1-bd0f-4315db7aad31 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:53.935654 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-846bc08f-86db-4fe1-bd0f-4315db7aad31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:53.956380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bc08f-86db-4fe1-bd0f-4315db7aad31 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:54.028379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf076e-d2ac-40d2-a958-3d8b76253d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:54.029441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf076e-d2ac-40d2-a958-3d8b76253d61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:54.029823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf076e-d2ac-40d2-a958-3d8b76253d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:54.030200 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf076e-d2ac-40d2-a958-3d8b76253d61 admin admin] PATH_INFO: `/v3/projects/7bb6bdd54d3d4a79a5300cd6d0bd2c1e` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:54.031897 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf076e-d2ac-40d2-a958-3d8b76253d61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bb6bdd54d3d4a79a5300cd6d0bd2c1e)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:54.035461 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf076e-d2ac-40d2-a958-3d8b76253d61 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:54.051763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:54.096622 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ffdf076e-d2ac-40d2-a958-3d8b76253d61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bb6bdd54d3d4a79a5300cd6d0bd2c1e', 'request_id': 'req-ffdf076e-d2ac-40d2-a958-3d8b76253d61'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:54.104408 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ffdf076e-d2ac-40d2-a958-3d8b76253d61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bb6bdd54d3d4a79a5300cd6d0bd2c1e', 'request_id': 'req-ffdf076e-d2ac-40d2-a958-3d8b76253d61'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:54.111930 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ffdf076e-d2ac-40d2-a958-3d8b76253d61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bb6bdd54d3d4a79a5300cd6d0bd2c1e', 'request_id': 'req-ffdf076e-d2ac-40d2-a958-3d8b76253d61'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:54.142315 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ffdf076e-d2ac-40d2-a958-3d8b76253d61 admin admin] The token cache is being invalidate because project 7bb6bdd54d3d4a79a5300cd6d0bd2c1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:39:54.142612 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ffdf076e-d2ac-40d2-a958-3d8b76253d61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:54.143301 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ffdf076e-d2ac-40d2-a958-3d8b76253d61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:39:54.144960 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1100/2232] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:39:53 2022] DELETE /identity/v3/projects/7bb6bdd54d3d4a79a5300cd6d0bd2c1e => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:54.150492 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca010f-624b-4506-be63-5af250bab7da None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:54.150815 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca010f-624b-4506-be63-5af250bab7da None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:54.151099 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca010f-624b-4506-be63-5af250bab7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:54.170004 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-cdca010f-624b-4506-be63-5af250bab7da None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:54.171324 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cdca010f-624b-4506-be63-5af250bab7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:54.216776 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:54.258909 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1133/2233] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:54.266518 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a990a64-10cd-4aa5-8803-3b45110490b7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:54.267083 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a990a64-10cd-4aa5-8803-3b45110490b7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:54.267492 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a990a64-10cd-4aa5-8803-3b45110490b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:54.288667 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1a990a64-10cd-4aa5-8803-3b45110490b7 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:54.290439 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1a990a64-10cd-4aa5-8803-3b45110490b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:54.300051 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1101/2234] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:54.304220 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca010f-624b-4506-be63-5af250bab7da None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:54.314776 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca010f-624b-4506-be63-5af250bab7da None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:54.321507 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027a4b6-22b9-4633-b744-1fb4242d7af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:54.322344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027a4b6-22b9-4633-b744-1fb4242d7af8 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:54.322746 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027a4b6-22b9-4633-b744-1fb4242d7af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:54.322952 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027a4b6-22b9-4633-b744-1fb4242d7af8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:54.323674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9027a4b6-22b9-4633-b744-1fb4242d7af8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:39:54.323982 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9027a4b6-22b9-4633-b744-1fb4242d7af8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:54.325797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9027a4b6-22b9-4633-b744-1fb4242d7af8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:54.340151 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1134/2235] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:39:54.344902 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a990a64-10cd-4aa5-8803-3b45110490b7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:54.357105 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a990a64-10cd-4aa5-8803-3b45110490b7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:54.364488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2537d81-bad3-4574-91e3-3f7a28236441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:54.365177 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2537d81-bad3-4574-91e3-3f7a28236441 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:54.365462 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2537d81-bad3-4574-91e3-3f7a28236441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:54.365748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2537d81-bad3-4574-91e3-3f7a28236441 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:54.366489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2537d81-bad3-4574-91e3-3f7a28236441 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:54.368510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2537d81-bad3-4574-91e3-3f7a28236441 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:54.400964 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1102/2236] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:39:54 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:39:54.402014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027a4b6-22b9-4633-b744-1fb4242d7af8 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:54.415496 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027a4b6-22b9-4633-b744-1fb4242d7af8 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:54.475891 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fae57-6945-4b08-8468-f8e5704890fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:54.476593 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fae57-6945-4b08-8468-f8e5704890fc admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:54.476908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fae57-6945-4b08-8468-f8e5704890fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:54.477700 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fae57-6945-4b08-8468-f8e5704890fc admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:54.478481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74fae57-6945-4b08-8468-f8e5704890fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:54.480644 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74fae57-6945-4b08-8468-f8e5704890fc admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:54.504395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1135/2237] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:39:54 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:39:54.509696 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2537d81-bad3-4574-91e3-3f7a28236441 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:54.527597 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2537d81-bad3-4574-91e3-3f7a28236441 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:54.534954 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c59bc1-ce86-49ac-9fe8-93ba66cc6689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:54.536009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c59bc1-ce86-49ac-9fe8-93ba66cc6689 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:54.536467 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c59bc1-ce86-49ac-9fe8-93ba66cc6689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:54.536916 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c59bc1-ce86-49ac-9fe8-93ba66cc6689 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:54.538153 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c59bc1-ce86-49ac-9fe8-93ba66cc6689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:54.541110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c59bc1-ce86-49ac-9fe8-93ba66cc6689 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:54.561827 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1103/2238] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:54.563076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fae57-6945-4b08-8468-f8e5704890fc admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:54.574103 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fae57-6945-4b08-8468-f8e5704890fc admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:54.578954 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd71c5-c8a9-43c7-a21f-bcbacf014985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:54.579653 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd71c5-c8a9-43c7-a21f-bcbacf014985 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:54.579942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd71c5-c8a9-43c7-a21f-bcbacf014985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:54.581205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd71c5-c8a9-43c7-a21f-bcbacf014985 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:54.581205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cd71c5-c8a9-43c7-a21f-bcbacf014985 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:54.582559 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cd71c5-c8a9-43c7-a21f-bcbacf014985 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:54.603885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1136/2239] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:39:54.608104 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c59bc1-ce86-49ac-9fe8-93ba66cc6689 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:54.618500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c59bc1-ce86-49ac-9fe8-93ba66cc6689 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:54.626271 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e016a6d-fe30-4bbf-a70c-5a7f25f40f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:54.627226 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e016a6d-fe30-4bbf-a70c-5a7f25f40f94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:54.627544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e016a6d-fe30-4bbf-a70c-5a7f25f40f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:54.627828 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e016a6d-fe30-4bbf-a70c-5a7f25f40f94 admin admin] PATH_INFO: `/v3/projects/9ee9f2493bbd40aa8608171470ebce72/users/385ac76a14c1462281619f839f4cefb4/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:54.643134 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e016a6d-fe30-4bbf-a70c-5a7f25f40f94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ee9f2493bbd40aa8608171470ebce72, user_id=385ac76a14c1462281619f839f4cefb4, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:54.645767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e016a6d-fe30-4bbf-a70c-5a7f25f40f94 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:54.661433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1104/2240] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:39:54 2022] PUT /identity/v3/projects/9ee9f2493bbd40aa8608171470ebce72/users/385ac76a14c1462281619f839f4cefb4/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:39:54.675534 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f66e78-bf71-4c85-a7fa-b5f072f246cd None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:54.675903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f66e78-bf71-4c85-a7fa-b5f072f246cd None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:54.676264 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f66e78-bf71-4c85-a7fa-b5f072f246cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:54.704752 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-e1f66e78-bf71-4c85-a7fa-b5f072f246cd None None] MFA Rules not processed for user `385ac76a14c1462281619f839f4cefb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:39:54.705996 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e1f66e78-bf71-4c85-a7fa-b5f072f246cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:54.779526 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1137/2241] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:39:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:39:54.796610 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e016a6d-fe30-4bbf-a70c-5a7f25f40f94 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:54.798381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7e016a6d-fe30-4bbf-a70c-5a7f25f40f94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:54.858102 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e016a6d-fe30-4bbf-a70c-5a7f25f40f94 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:54.861735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1bc9e-c323-426e-b15f-2ade34a40061 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:54.862388 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1bc9e-c323-426e-b15f-2ade34a40061 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:54.862705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1bc9e-c323-426e-b15f-2ade34a40061 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:54.862990 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1bc9e-c323-426e-b15f-2ade34a40061 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:54.871776 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1bc9e-c323-426e-b15f-2ade34a40061 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:54.873205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1bc9e-c323-426e-b15f-2ade34a40061 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:54.881904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1105/2242] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:39:55.507620 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f66e78-bf71-4c85-a7fa-b5f072f246cd None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:39:55.517275 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f66e78-bf71-4c85-a7fa-b5f072f246cd None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:39:55.520830 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509fe9aa-d11c-4bb0-bd0d-7fc6b646a14b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:39:55.521467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509fe9aa-d11c-4bb0-bd0d-7fc6b646a14b service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:39:55.521768 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509fe9aa-d11c-4bb0-bd0d-7fc6b646a14b service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:39:55.522026 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509fe9aa-d11c-4bb0-bd0d-7fc6b646a14b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:39:55.523817 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-509fe9aa-d11c-4bb0-bd0d-7fc6b646a14b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:39:55.562826 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:39:55.582788 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509fe9aa-d11c-4bb0-bd0d-7fc6b646a14b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:39:55.584241 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509fe9aa-d11c-4bb0-bd0d-7fc6b646a14b service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:39:55.593528 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1138/2243] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:39:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:40:00.793835 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1bc9e-c323-426e-b15f-2ade34a40061 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:00.803031 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1bc9e-c323-426e-b15f-2ade34a40061 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:00.806664 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768310a-60b7-402b-a096-ac8408a73083 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:00.807310 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768310a-60b7-402b-a096-ac8408a73083 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:00.807647 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768310a-60b7-402b-a096-ac8408a73083 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:00.807948 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768310a-60b7-402b-a096-ac8408a73083 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:00.809768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7768310a-60b7-402b-a096-ac8408a73083 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:00.851309 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:00.877453 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7768310a-60b7-402b-a096-ac8408a73083 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:00.878953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7768310a-60b7-402b-a096-ac8408a73083 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:00.889208 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1106/2244] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:40:00 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:40:05.589518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=10 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:10.286821 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509fe9aa-d11c-4bb0-bd0d-7fc6b646a14b service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:10.371485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509fe9aa-d11c-4bb0-bd0d-7fc6b646a14b service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:10.379295 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07cd4de-4a4f-4850-b0de-548ed1d366d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:10.380417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cd4de-4a4f-4850-b0de-548ed1d366d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:10.380884 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cd4de-4a4f-4850-b0de-548ed1d366d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:10.381286 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cd4de-4a4f-4850-b0de-548ed1d366d6 admin admin] PATH_INFO: `/v3/users/385ac76a14c1462281619f839f4cefb4` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:10.382682 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07cd4de-4a4f-4850-b0de-548ed1d366d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=385ac76a14c1462281619f839f4cefb4)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:10.384971 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07cd4de-4a4f-4850-b0de-548ed1d366d6 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:10.448712 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f07cd4de-4a4f-4850-b0de-548ed1d366d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '385ac76a14c1462281619f839f4cefb4'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:10.458075 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f07cd4de-4a4f-4850-b0de-548ed1d366d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '385ac76a14c1462281619f839f4cefb4'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:10.459161 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f07cd4de-4a4f-4850-b0de-548ed1d366d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '385ac76a14c1462281619f839f4cefb4'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:10.474746 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f07cd4de-4a4f-4850-b0de-548ed1d366d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '385ac76a14c1462281619f839f4cefb4'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:10.475683 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f07cd4de-4a4f-4850-b0de-548ed1d366d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '385ac76a14c1462281619f839f4cefb4'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:10.494470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1139/2245] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:40:10 2022] DELETE /identity/v3/users/385ac76a14c1462281619f839f4cefb4 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:40:10.498995 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac465d-db09-4555-b207-72a4e02cced7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:10.498995 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac465d-db09-4555-b207-72a4e02cced7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:10.498995 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac465d-db09-4555-b207-72a4e02cced7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:10.520012 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-26ac465d-db09-4555-b207-72a4e02cced7 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:10.521587 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-26ac465d-db09-4555-b207-72a4e02cced7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:10.587495 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1107/2246] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:10.597085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07cd4de-4a4f-4850-b0de-548ed1d366d6 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:10.598509 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f07cd4de-4a4f-4850-b0de-548ed1d366d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:10.662037 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07cd4de-4a4f-4850-b0de-548ed1d366d6 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:10.665658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ae18c-c64b-4163-ad85-b763f9778a21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:10.666304 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ae18c-c64b-4163-ad85-b763f9778a21 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:10.666586 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ae18c-c64b-4163-ad85-b763f9778a21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:10.666965 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ae18c-c64b-4163-ad85-b763f9778a21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:10.676433 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ae18c-c64b-4163-ad85-b763f9778a21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:10.677796 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ae18c-c64b-4163-ad85-b763f9778a21 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:10.687736 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1140/2247] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:40:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:40:10.958279 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac465d-db09-4555-b207-72a4e02cced7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:10.959731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-26ac465d-db09-4555-b207-72a4e02cced7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:10.974462 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac465d-db09-4555-b207-72a4e02cced7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:10.981166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e24f28-6fe2-4292-9d2d-3720b7cba224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:10.982132 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e24f28-6fe2-4292-9d2d-3720b7cba224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:10.982408 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e24f28-6fe2-4292-9d2d-3720b7cba224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:10.982796 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e24f28-6fe2-4292-9d2d-3720b7cba224 admin admin] PATH_INFO: `/v3/projects/9ee9f2493bbd40aa8608171470ebce72` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:10.984109 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e24f28-6fe2-4292-9d2d-3720b7cba224 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ee9f2493bbd40aa8608171470ebce72)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:10.986161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e24f28-6fe2-4292-9d2d-3720b7cba224 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:11.077032 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-76e24f28-6fe2-4292-9d2d-3720b7cba224 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ee9f2493bbd40aa8608171470ebce72', 'request_id': 'req-76e24f28-6fe2-4292-9d2d-3720b7cba224'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:11.085928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-76e24f28-6fe2-4292-9d2d-3720b7cba224 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ee9f2493bbd40aa8608171470ebce72', 'request_id': 'req-76e24f28-6fe2-4292-9d2d-3720b7cba224'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:11.093113 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-76e24f28-6fe2-4292-9d2d-3720b7cba224 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ee9f2493bbd40aa8608171470ebce72', 'request_id': 'req-76e24f28-6fe2-4292-9d2d-3720b7cba224'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:11.093260 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ae18c-c64b-4163-ad85-b763f9778a21 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:11.094766 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f37ae18c-c64b-4163-ad85-b763f9778a21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:11.107999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-76e24f28-6fe2-4292-9d2d-3720b7cba224 admin admin] The token cache is being invalidate because project 9ee9f2493bbd40aa8608171470ebce72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:40:11.108331 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ae18c-c64b-4163-ad85-b763f9778a21 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:11.108468 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-76e24f28-6fe2-4292-9d2d-3720b7cba224 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:11.108982 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-76e24f28-6fe2-4292-9d2d-3720b7cba224 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:11.110933 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1108/2248] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:40:10 2022] DELETE /identity/v3/projects/9ee9f2493bbd40aa8608171470ebce72 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:40:11.115776 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019fd64-cd85-4f44-8e3d-8e969660fefb None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:11.116069 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019fd64-cd85-4f44-8e3d-8e969660fefb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:11.116345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019fd64-cd85-4f44-8e3d-8e969660fefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:11.131469 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:11.149066 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d019fd64-cd85-4f44-8e3d-8e969660fefb None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:11.151182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d019fd64-cd85-4f44-8e3d-8e969660fefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:11.175391 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e21ff-a9f0-4d1b-86cc-7a56246ba638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:11.176274 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e21ff-a9f0-4d1b-86cc-7a56246ba638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:11.176553 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e21ff-a9f0-4d1b-86cc-7a56246ba638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:11.176826 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e21ff-a9f0-4d1b-86cc-7a56246ba638 admin admin] PATH_INFO: `/v3/users/ab1506823ed5423d8056af2818140672` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:11.178015 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e21ff-a9f0-4d1b-86cc-7a56246ba638 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab1506823ed5423d8056af2818140672)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:11.179968 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e21ff-a9f0-4d1b-86cc-7a56246ba638 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:11.217457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:11.233369 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1109/2249] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:11.242145 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8deb0b-fdbe-4d95-b0dd-ce60180e04fa None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:11.242713 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8deb0b-fdbe-4d95-b0dd-ce60180e04fa None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:11.243142 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8deb0b-fdbe-4d95-b0dd-ce60180e04fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:11.248422 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ac6e21ff-a9f0-4d1b-86cc-7a56246ba638 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab1506823ed5423d8056af2818140672'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:11.259252 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ac6e21ff-a9f0-4d1b-86cc-7a56246ba638 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab1506823ed5423d8056af2818140672'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:11.259999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ac6e21ff-a9f0-4d1b-86cc-7a56246ba638 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab1506823ed5423d8056af2818140672'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:11.274327 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4a8deb0b-fdbe-4d95-b0dd-ce60180e04fa None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:11.275645 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4a8deb0b-fdbe-4d95-b0dd-ce60180e04fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:11.281322 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ac6e21ff-a9f0-4d1b-86cc-7a56246ba638 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab1506823ed5423d8056af2818140672'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:11.282006 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ac6e21ff-a9f0-4d1b-86cc-7a56246ba638 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab1506823ed5423d8056af2818140672'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:11.285096 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1110/2250] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:11.289049 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8deb0b-fdbe-4d95-b0dd-ce60180e04fa None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:11.296509 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1141/2251] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:40:11 2022] DELETE /identity/v3/users/ab1506823ed5423d8056af2818140672 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:40:11.299932 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8deb0b-fdbe-4d95-b0dd-ce60180e04fa None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:11.302801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c4248-9641-4eb6-9524-a0a5e8c11e95 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:11.304799 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c4248-9641-4eb6-9524-a0a5e8c11e95 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:11.305260 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c4248-9641-4eb6-9524-a0a5e8c11e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:11.325906 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-d67c4248-9641-4eb6-9524-a0a5e8c11e95 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:11.327867 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d67c4248-9641-4eb6-9524-a0a5e8c11e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:11.376057 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e6088-b3e1-4c7e-90ec-796db545f057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:11.376835 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e6088-b3e1-4c7e-90ec-796db545f057 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:11.377190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e6088-b3e1-4c7e-90ec-796db545f057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:11.377562 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e6088-b3e1-4c7e-90ec-796db545f057 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:11.378370 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e6088-b3e1-4c7e-90ec-796db545f057 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:40:11.378771 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e6088-b3e1-4c7e-90ec-796db545f057 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:11.380462 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e6088-b3e1-4c7e-90ec-796db545f057 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:11.392054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1142/2252] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:40:11.395267 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1111/2253] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:40:11.397372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c4248-9641-4eb6-9524-a0a5e8c11e95 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:11.402901 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e6088-b3e1-4c7e-90ec-796db545f057 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:11.404916 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e39e6088-b3e1-4c7e-90ec-796db545f057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:11.408021 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c4248-9641-4eb6-9524-a0a5e8c11e95 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:11.414606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6f5bd-e3c8-49a0-97f1-1f523c00e0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:11.415301 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6f5bd-e3c8-49a0-97f1-1f523c00e0e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:11.415586 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6f5bd-e3c8-49a0-97f1-1f523c00e0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:11.415860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6f5bd-e3c8-49a0-97f1-1f523c00e0e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:11.416737 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a6f5bd-e3c8-49a0-97f1-1f523c00e0e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:11.419414 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a6f5bd-e3c8-49a0-97f1-1f523c00e0e5 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:11.448458 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1143/2254] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:40:11 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:40:11.453168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6f5bd-e3c8-49a0-97f1-1f523c00e0e5 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:11.467630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e6088-b3e1-4c7e-90ec-796db545f057 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:11.484579 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:11.533912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab8355-e54c-47c2-9cb3-6befaf1f3f05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:11.535239 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab8355-e54c-47c2-9cb3-6befaf1f3f05 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:11.535685 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab8355-e54c-47c2-9cb3-6befaf1f3f05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:11.536116 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab8355-e54c-47c2-9cb3-6befaf1f3f05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:11.552143 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6f5bd-e3c8-49a0-97f1-1f523c00e0e5 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:11.564324 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9953f15-c985-42b8-8de9-c6439835e08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:11.565818 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab8355-e54c-47c2-9cb3-6befaf1f3f05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:11.570439 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab8355-e54c-47c2-9cb3-6befaf1f3f05 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:11.574504 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9953f15-c985-42b8-8de9-c6439835e08b admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:11.574828 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9953f15-c985-42b8-8de9-c6439835e08b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:11.575164 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9953f15-c985-42b8-8de9-c6439835e08b admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:11.575892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9953f15-c985-42b8-8de9-c6439835e08b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:11.577864 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9953f15-c985-42b8-8de9-c6439835e08b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:11.587128 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1112/2255] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:40:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:40:11.608340 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1144/2256] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:40:11 2022] POST /identity/v3/users => generated 288 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:40:11.612461 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab8355-e54c-47c2-9cb3-6befaf1f3f05 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:11.624614 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab8355-e54c-47c2-9cb3-6befaf1f3f05 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:11.630570 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1951c-3669-463c-bf1d-09fe28c6939c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:11.631438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1951c-3669-463c-bf1d-09fe28c6939c admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:11.631818 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1951c-3669-463c-bf1d-09fe28c6939c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:11.632192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1951c-3669-463c-bf1d-09fe28c6939c admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:11.634672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1951c-3669-463c-bf1d-09fe28c6939c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:11.634922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1951c-3669-463c-bf1d-09fe28c6939c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:11.649943 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1113/2257] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:40:11.651042 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9953f15-c985-42b8-8de9-c6439835e08b admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:11.661631 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9953f15-c985-42b8-8de9-c6439835e08b admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:11.667761 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037f0aa-df72-4c48-a4b9-541149c0c3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:11.669216 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037f0aa-df72-4c48-a4b9-541149c0c3b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:11.669327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037f0aa-df72-4c48-a4b9-541149c0c3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:11.669538 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037f0aa-df72-4c48-a4b9-541149c0c3b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:11.670278 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0037f0aa-df72-4c48-a4b9-541149c0c3b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:11.671974 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0037f0aa-df72-4c48-a4b9-541149c0c3b3 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:11.686702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1145/2258] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:40:11.688163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1951c-3669-463c-bf1d-09fe28c6939c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:11.699154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1951c-3669-463c-bf1d-09fe28c6939c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:11.704517 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda6159-5727-4042-a80d-df7f28632b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:11.705505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda6159-5727-4042-a80d-df7f28632b55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:11.705896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda6159-5727-4042-a80d-df7f28632b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:11.706283 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda6159-5727-4042-a80d-df7f28632b55 admin admin] PATH_INFO: `/v3/projects/7dadc6972be242a48c64084726085851/users/ab5d06b0a3324dab903f485342590db9/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:11.716399 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f988050-c101-4345-b5e8-f2236314bf45 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:11.716698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f988050-c101-4345-b5e8-f2236314bf45 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:11.717028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f988050-c101-4345-b5e8-f2236314bf45 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:11.721531 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1146/2259] 10.222.0.67 () {54 vars in 921 bytes} [Tue May 31 14:40:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:40:11.728813 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcda6159-5727-4042-a80d-df7f28632b55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dadc6972be242a48c64084726085851, user_id=ab5d06b0a3324dab903f485342590db9, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:11.732465 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcda6159-5727-4042-a80d-df7f28632b55 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:11.754259 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1114/2260] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:40:11 2022] PUT /identity/v3/projects/7dadc6972be242a48c64084726085851/users/ab5d06b0a3324dab903f485342590db9/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:40:11.764816 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ef0e1-1fd4-49b0-9ff7-122a41048eb2 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:11.765226 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ef0e1-1fd4-49b0-9ff7-122a41048eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:11.765651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ef0e1-1fd4-49b0-9ff7-122a41048eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:11.768228 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea7ce6-b548-46da-a77d-e34235c08717 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:11.769143 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea7ce6-b548-46da-a77d-e34235c08717 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:11.769599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea7ce6-b548-46da-a77d-e34235c08717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:11.798330 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a20ef0e1-1fd4-49b0-9ff7-122a41048eb2 None None] MFA Rules not processed for user `6c1c13515a8b41a781478dc65e6a88b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:11.799926 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a20ef0e1-1fd4-49b0-9ff7-122a41048eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:11.800906 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:11.830368 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3eea7ce6-b548-46da-a77d-e34235c08717 None None] MFA Rules not processed for user `ab5d06b0a3324dab903f485342590db9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:11.833044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3eea7ce6-b548-46da-a77d-e34235c08717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:11.905515 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1147/2261] 10.222.0.67 () {58 vars in 1018 bytes} [Tue May 31 14:40:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:40:11.916170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ef0e1-1fd4-49b0-9ff7-122a41048eb2 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:11.918204 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a20ef0e1-1fd4-49b0-9ff7-122a41048eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:11.931899 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1115/2262] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:11.932857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea7ce6-b548-46da-a77d-e34235c08717 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:11.934176 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3eea7ce6-b548-46da-a77d-e34235c08717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:11.953207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea7ce6-b548-46da-a77d-e34235c08717 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:11.960638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc1489-dcbf-477f-b759-bae98e6fe4e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c1c13515a8b41a781478dc65e6a88b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:11.961240 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc1489-dcbf-477f-b759-bae98e6fe4e9 service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:11.961528 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc1489-dcbf-477f-b759-bae98e6fe4e9 service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:11.961807 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc1489-dcbf-477f-b759-bae98e6fe4e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:11.971441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc1489-dcbf-477f-b759-bae98e6fe4e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:11.973042 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc1489-dcbf-477f-b759-bae98e6fe4e9 service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:11.993364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1116/2263] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:40:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:40:11.994568 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc1489-dcbf-477f-b759-bae98e6fe4e9 service placement] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:11.996440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d9dc1489-dcbf-477f-b759-bae98e6fe4e9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:12.013966 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ef0e1-1fd4-49b0-9ff7-122a41048eb2 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:12.019510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269ec7c-7287-4cdc-8a84-43cd0cb1ed0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:12.020638 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269ec7c-7287-4cdc-8a84-43cd0cb1ed0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:12.021024 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269ec7c-7287-4cdc-8a84-43cd0cb1ed0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:12.021443 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269ec7c-7287-4cdc-8a84-43cd0cb1ed0d admin admin] PATH_INFO: `/v3/users/ea126c816da740dda9622f7bfa10dd16` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:12.022900 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8269ec7c-7287-4cdc-8a84-43cd0cb1ed0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea126c816da740dda9622f7bfa10dd16)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:12.025738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8269ec7c-7287-4cdc-8a84-43cd0cb1ed0d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:12.074415 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:12.078743 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc1489-dcbf-477f-b759-bae98e6fe4e9 service placement] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:12.082723 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91573c-765d-4196-aeb4-8e932ec692b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:12.083432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91573c-765d-4196-aeb4-8e932ec692b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:12.083723 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91573c-765d-4196-aeb4-8e932ec692b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:12.084001 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91573c-765d-4196-aeb4-8e932ec692b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:12.084671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8269ec7c-7287-4cdc-8a84-43cd0cb1ed0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea126c816da740dda9622f7bfa10dd16'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:12.094803 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8269ec7c-7287-4cdc-8a84-43cd0cb1ed0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea126c816da740dda9622f7bfa10dd16'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:12.095547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8269ec7c-7287-4cdc-8a84-43cd0cb1ed0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea126c816da740dda9622f7bfa10dd16'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:12.095690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91573c-765d-4196-aeb4-8e932ec692b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:12.097292 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91573c-765d-4196-aeb4-8e932ec692b9 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:12.110154 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1117/2264] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:40:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:40:12.111072 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91573c-765d-4196-aeb4-8e932ec692b9 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:12.112526 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9f91573c-765d-4196-aeb4-8e932ec692b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:12.113281 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8269ec7c-7287-4cdc-8a84-43cd0cb1ed0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea126c816da740dda9622f7bfa10dd16'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:12.114394 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8269ec7c-7287-4cdc-8a84-43cd0cb1ed0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea126c816da740dda9622f7bfa10dd16'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:12.128920 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91573c-765d-4196-aeb4-8e932ec692b9 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:12.135595 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503d77c-49a0-4536-b108-5dc6c1527983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:12.136575 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503d77c-49a0-4536-b108-5dc6c1527983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:12.138619 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503d77c-49a0-4536-b108-5dc6c1527983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:12.138907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503d77c-49a0-4536-b108-5dc6c1527983 admin admin] PATH_INFO: `/v3/projects/88361a67b4694d48a98f01350a1c35eb` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:12.140421 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503d77c-49a0-4536-b108-5dc6c1527983 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88361a67b4694d48a98f01350a1c35eb)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:12.142556 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503d77c-49a0-4536-b108-5dc6c1527983 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:12.150358 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1148/2265] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:40:11 2022] DELETE /identity/v3/users/ea126c816da740dda9622f7bfa10dd16 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:40:12.208246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a503d77c-49a0-4536-b108-5dc6c1527983 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88361a67b4694d48a98f01350a1c35eb', 'request_id': 'req-a503d77c-49a0-4536-b108-5dc6c1527983'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:12.214414 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a503d77c-49a0-4536-b108-5dc6c1527983 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88361a67b4694d48a98f01350a1c35eb', 'request_id': 'req-a503d77c-49a0-4536-b108-5dc6c1527983'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:12.221706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a503d77c-49a0-4536-b108-5dc6c1527983 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88361a67b4694d48a98f01350a1c35eb', 'request_id': 'req-a503d77c-49a0-4536-b108-5dc6c1527983'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:12.242604 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a503d77c-49a0-4536-b108-5dc6c1527983 admin admin] The token cache is being invalidate because project 88361a67b4694d48a98f01350a1c35eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:40:12.243221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a503d77c-49a0-4536-b108-5dc6c1527983 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:12.243961 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a503d77c-49a0-4536-b108-5dc6c1527983 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:12.246028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1118/2266] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:40:12 2022] DELETE /identity/v3/projects/88361a67b4694d48a98f01350a1c35eb => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:40:12.250414 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269ec7c-7287-4cdc-8a84-43cd0cb1ed0d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:12.252447 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8269ec7c-7287-4cdc-8a84-43cd0cb1ed0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:12.357625 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269ec7c-7287-4cdc-8a84-43cd0cb1ed0d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:12.373168 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503d77c-49a0-4536-b108-5dc6c1527983 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:12.374875 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a503d77c-49a0-4536-b108-5dc6c1527983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:12.395778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:12.452113 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe0205-75a5-434d-a473-3c8cc7a7002a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:12.453222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe0205-75a5-434d-a473-3c8cc7a7002a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:12.453815 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe0205-75a5-434d-a473-3c8cc7a7002a admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:12.453815 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe0205-75a5-434d-a473-3c8cc7a7002a admin admin] PATH_INFO: `/v3/users/c34f02289ccf43488994f98d2ab50048` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:12.455410 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe0205-75a5-434d-a473-3c8cc7a7002a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c34f02289ccf43488994f98d2ab50048)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:12.457830 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe0205-75a5-434d-a473-3c8cc7a7002a admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:12.481750 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503d77c-49a0-4536-b108-5dc6c1527983 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:12.487035 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd77fb-bf2e-4787-813e-edf56408b108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:12.488107 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd77fb-bf2e-4787-813e-edf56408b108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:12.488823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd77fb-bf2e-4787-813e-edf56408b108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:12.488823 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd77fb-bf2e-4787-813e-edf56408b108 admin admin] PATH_INFO: `/v3/projects/0300307fde534d49a050041217cfe6e1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:12.490112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd77fb-bf2e-4787-813e-edf56408b108 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0300307fde534d49a050041217cfe6e1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:12.492530 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd77fb-bf2e-4787-813e-edf56408b108 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:12.529446 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0cfe0205-75a5-434d-a473-3c8cc7a7002a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c34f02289ccf43488994f98d2ab50048'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:12.537771 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0cfe0205-75a5-434d-a473-3c8cc7a7002a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c34f02289ccf43488994f98d2ab50048'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:12.538640 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0cfe0205-75a5-434d-a473-3c8cc7a7002a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c34f02289ccf43488994f98d2ab50048'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:12.549184 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:12.552295 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-07fd77fb-bf2e-4787-813e-edf56408b108 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0300307fde534d49a050041217cfe6e1', 'request_id': 'req-07fd77fb-bf2e-4787-813e-edf56408b108'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:12.557499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0cfe0205-75a5-434d-a473-3c8cc7a7002a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c34f02289ccf43488994f98d2ab50048'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:12.558913 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0cfe0205-75a5-434d-a473-3c8cc7a7002a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c34f02289ccf43488994f98d2ab50048'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:12.561215 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-07fd77fb-bf2e-4787-813e-edf56408b108 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0300307fde534d49a050041217cfe6e1', 'request_id': 'req-07fd77fb-bf2e-4787-813e-edf56408b108'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:12.571077 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-07fd77fb-bf2e-4787-813e-edf56408b108 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0300307fde534d49a050041217cfe6e1', 'request_id': 'req-07fd77fb-bf2e-4787-813e-edf56408b108'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:12.574894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1149/2267] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:40:12 2022] DELETE /identity/v3/users/c34f02289ccf43488994f98d2ab50048 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:40:12.591057 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-07fd77fb-bf2e-4787-813e-edf56408b108 admin admin] The token cache is being invalidate because project 0300307fde534d49a050041217cfe6e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:40:12.591494 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-07fd77fb-bf2e-4787-813e-edf56408b108 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:12.592416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-07fd77fb-bf2e-4787-813e-edf56408b108 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:12.594695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1119/2268] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:40:12 2022] DELETE /identity/v3/projects/0300307fde534d49a050041217cfe6e1 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:40:12.599277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319716c8-74d9-48f0-ab5b-88add09fc69a None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:12.599569 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319716c8-74d9-48f0-ab5b-88add09fc69a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:12.599850 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319716c8-74d9-48f0-ab5b-88add09fc69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:12.619029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-319716c8-74d9-48f0-ab5b-88add09fc69a None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:12.620396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-319716c8-74d9-48f0-ab5b-88add09fc69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:12.683084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1150/2269] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:12.692408 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083503e-2584-41df-9946-e50fca1b0e91 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:12.692705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083503e-2584-41df-9946-e50fca1b0e91 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:12.692996 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083503e-2584-41df-9946-e50fca1b0e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:12.712885 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8083503e-2584-41df-9946-e50fca1b0e91 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:12.714117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8083503e-2584-41df-9946-e50fca1b0e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:12.722281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1120/2270] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:12.726529 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319716c8-74d9-48f0-ab5b-88add09fc69a None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:12.737152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319716c8-74d9-48f0-ab5b-88add09fc69a None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:12.742164 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac5136-51cb-4796-a44a-e27e1ced2283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:12.743087 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac5136-51cb-4796-a44a-e27e1ced2283 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:12.743382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac5136-51cb-4796-a44a-e27e1ced2283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:12.743664 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac5136-51cb-4796-a44a-e27e1ced2283 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:12.744409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac5136-51cb-4796-a44a-e27e1ced2283 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:40:12.744724 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac5136-51cb-4796-a44a-e27e1ced2283 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:12.747993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac5136-51cb-4796-a44a-e27e1ced2283 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:12.763103 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1151/2271] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:40:12.767513 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083503e-2584-41df-9946-e50fca1b0e91 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:12.777239 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083503e-2584-41df-9946-e50fca1b0e91 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:12.782034 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac98c8b-6854-4607-baf3-c4066ff0f692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:12.782856 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac98c8b-6854-4607-baf3-c4066ff0f692 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:12.783206 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac98c8b-6854-4607-baf3-c4066ff0f692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:12.783556 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac98c8b-6854-4607-baf3-c4066ff0f692 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:12.784335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac98c8b-6854-4607-baf3-c4066ff0f692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:12.786321 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac98c8b-6854-4607-baf3-c4066ff0f692 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:12.815311 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1121/2272] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:40:12 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:40:12.819543 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac5136-51cb-4796-a44a-e27e1ced2283 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:12.830345 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac5136-51cb-4796-a44a-e27e1ced2283 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:12.901777 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c22ad9f-8cad-4990-9e31-8cf968f229eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:12.902604 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22ad9f-8cad-4990-9e31-8cf968f229eb admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:12.903029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22ad9f-8cad-4990-9e31-8cf968f229eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:12.903384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22ad9f-8cad-4990-9e31-8cf968f229eb admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:12.904177 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c22ad9f-8cad-4990-9e31-8cf968f229eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:12.906250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c22ad9f-8cad-4990-9e31-8cf968f229eb admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:12.930890 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1152/2273] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:40:12 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:40:12.935709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac98c8b-6854-4607-baf3-c4066ff0f692 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:12.947157 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac98c8b-6854-4607-baf3-c4066ff0f692 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:12.952392 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab11f4c-4662-48f1-9290-b502c4c4b8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:12.953077 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab11f4c-4662-48f1-9290-b502c4c4b8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:12.953983 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab11f4c-4662-48f1-9290-b502c4c4b8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:12.954419 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab11f4c-4662-48f1-9290-b502c4c4b8c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:12.955208 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab11f4c-4662-48f1-9290-b502c4c4b8c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:12.956829 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab11f4c-4662-48f1-9290-b502c4c4b8c2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:12.974351 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1122/2274] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:40:12.978788 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c22ad9f-8cad-4990-9e31-8cf968f229eb admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:12.995103 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c22ad9f-8cad-4990-9e31-8cf968f229eb admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:13.004062 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a29c1f-1548-4631-b2da-c00ff3c09ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:13.004747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a29c1f-1548-4631-b2da-c00ff3c09ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:13.005034 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a29c1f-1548-4631-b2da-c00ff3c09ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:13.005312 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a29c1f-1548-4631-b2da-c00ff3c09ad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:13.006048 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a29c1f-1548-4631-b2da-c00ff3c09ad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:13.007698 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a29c1f-1548-4631-b2da-c00ff3c09ad2 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:13.019658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1153/2275] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:40:13.023835 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab11f4c-4662-48f1-9290-b502c4c4b8c2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:13.034948 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab11f4c-4662-48f1-9290-b502c4c4b8c2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:13.039646 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72260b3c-d451-44a4-bb75-6b08148f281f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:13.040549 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72260b3c-d451-44a4-bb75-6b08148f281f admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:13.040842 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72260b3c-d451-44a4-bb75-6b08148f281f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:13.041126 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72260b3c-d451-44a4-bb75-6b08148f281f admin admin] PATH_INFO: `/v3/projects/502541429884445e8426837b3fd0d75e/users/32e6ace174a840f78c8d570719c3f631/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:13.055640 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72260b3c-d451-44a4-bb75-6b08148f281f admin admin] RBAC: Authorizing `identity:create_grant(project_id=502541429884445e8426837b3fd0d75e, user_id=32e6ace174a840f78c8d570719c3f631, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:13.058281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72260b3c-d451-44a4-bb75-6b08148f281f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:13.072118 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1123/2276] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:40:13 2022] PUT /identity/v3/projects/502541429884445e8426837b3fd0d75e/users/32e6ace174a840f78c8d570719c3f631/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:40:13.078164 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae5bec-40ea-420e-9314-9f1bfb496e03 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:13.078478 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae5bec-40ea-420e-9314-9f1bfb496e03 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:13.078795 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae5bec-40ea-420e-9314-9f1bfb496e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:13.092003 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72260b3c-d451-44a4-bb75-6b08148f281f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:13.094221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-72260b3c-d451-44a4-bb75-6b08148f281f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:13.098775 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a5ae5bec-40ea-420e-9314-9f1bfb496e03 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:13.100166 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a5ae5bec-40ea-420e-9314-9f1bfb496e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:13.148784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:13.159597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:13.177724 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72260b3c-d451-44a4-bb75-6b08148f281f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:13.181267 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1154/2277] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:13.186524 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b723b7-3652-41e1-bf6e-3d4377fd590f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:13.187449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b723b7-3652-41e1-bf6e-3d4377fd590f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:13.187744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b723b7-3652-41e1-bf6e-3d4377fd590f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:13.188021 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b723b7-3652-41e1-bf6e-3d4377fd590f admin admin] PATH_INFO: `/v3/projects/5b4fda417c7f4472b7d919554e1a153e` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:13.189500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b723b7-3652-41e1-bf6e-3d4377fd590f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b4fda417c7f4472b7d919554e1a153e)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:13.191564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae5bec-40ea-420e-9314-9f1bfb496e03 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:13.191735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b723b7-3652-41e1-bf6e-3d4377fd590f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:13.193287 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a5ae5bec-40ea-420e-9314-9f1bfb496e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:13.241370 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e5b723b7-3652-41e1-bf6e-3d4377fd590f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b4fda417c7f4472b7d919554e1a153e', 'request_id': 'req-e5b723b7-3652-41e1-bf6e-3d4377fd590f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:13.247503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e5b723b7-3652-41e1-bf6e-3d4377fd590f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b4fda417c7f4472b7d919554e1a153e', 'request_id': 'req-e5b723b7-3652-41e1-bf6e-3d4377fd590f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:13.252090 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae5bec-40ea-420e-9314-9f1bfb496e03 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:13.253781 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e5b723b7-3652-41e1-bf6e-3d4377fd590f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b4fda417c7f4472b7d919554e1a153e', 'request_id': 'req-e5b723b7-3652-41e1-bf6e-3d4377fd590f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:13.267827 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e5b723b7-3652-41e1-bf6e-3d4377fd590f admin admin] The token cache is being invalidate because project 5b4fda417c7f4472b7d919554e1a153e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:40:13.268122 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e5b723b7-3652-41e1-bf6e-3d4377fd590f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:13.268769 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e5b723b7-3652-41e1-bf6e-3d4377fd590f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:13.270353 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1124/2278] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:40:13 2022] DELETE /identity/v3/projects/5b4fda417c7f4472b7d919554e1a153e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:40:13.274433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b723b7-3652-41e1-bf6e-3d4377fd590f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:13.275769 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e5b723b7-3652-41e1-bf6e-3d4377fd590f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:13.301430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ec471-9a2d-4b41-915d-d7950170c51f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:13.302090 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ec471-9a2d-4b41-915d-d7950170c51f service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:13.302401 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ec471-9a2d-4b41-915d-d7950170c51f service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:13.302701 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ec471-9a2d-4b41-915d-d7950170c51f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:13.304715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5d2ec471-9a2d-4b41-915d-d7950170c51f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:13.343255 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b723b7-3652-41e1-bf6e-3d4377fd590f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:13.354481 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456462de-6e3f-4ed8-94ae-a49c1f6c2f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:13.355523 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456462de-6e3f-4ed8-94ae-a49c1f6c2f30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:13.355815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456462de-6e3f-4ed8-94ae-a49c1f6c2f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:13.356247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456462de-6e3f-4ed8-94ae-a49c1f6c2f30 admin admin] PATH_INFO: `/v3/users/86123a2a28cc43eca7d8c20fa1078857` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:13.357864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456462de-6e3f-4ed8-94ae-a49c1f6c2f30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86123a2a28cc43eca7d8c20fa1078857)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:13.359809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456462de-6e3f-4ed8-94ae-a49c1f6c2f30 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:13.380802 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ec471-9a2d-4b41-915d-d7950170c51f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:13.382265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ec471-9a2d-4b41-915d-d7950170c51f service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:13.394499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1155/2279] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:40:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:40:13.412950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-456462de-6e3f-4ed8-94ae-a49c1f6c2f30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86123a2a28cc43eca7d8c20fa1078857'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:13.421066 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-456462de-6e3f-4ed8-94ae-a49c1f6c2f30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86123a2a28cc43eca7d8c20fa1078857'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:13.428268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-456462de-6e3f-4ed8-94ae-a49c1f6c2f30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86123a2a28cc43eca7d8c20fa1078857'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:13.433618 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:13.443368 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-456462de-6e3f-4ed8-94ae-a49c1f6c2f30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86123a2a28cc43eca7d8c20fa1078857'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:13.444310 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-456462de-6e3f-4ed8-94ae-a49c1f6c2f30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86123a2a28cc43eca7d8c20fa1078857'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:13.450569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1125/2280] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:40:13 2022] DELETE /identity/v3/users/86123a2a28cc43eca7d8c20fa1078857 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:40:13.757050 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ec471-9a2d-4b41-915d-d7950170c51f service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:13.772396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ec471-9a2d-4b41-915d-d7950170c51f service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:13.844899 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:13.855685 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e03d49-5dfd-4e99-b397-08ef775c2597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:13.857843 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e03d49-5dfd-4e99-b397-08ef775c2597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:13.858151 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e03d49-5dfd-4e99-b397-08ef775c2597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:13.858442 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e03d49-5dfd-4e99-b397-08ef775c2597 admin admin] PATH_INFO: `/v3/projects/338b138d409f4447b823202c3c028ae1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:13.859863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e03d49-5dfd-4e99-b397-08ef775c2597 admin admin] RBAC: Authorizing `identity:delete_project(project_id=338b138d409f4447b823202c3c028ae1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:13.862101 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e03d49-5dfd-4e99-b397-08ef775c2597 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:13.914547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-01e03d49-5dfd-4e99-b397-08ef775c2597 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '338b138d409f4447b823202c3c028ae1', 'request_id': 'req-01e03d49-5dfd-4e99-b397-08ef775c2597'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:13.920324 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-01e03d49-5dfd-4e99-b397-08ef775c2597 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '338b138d409f4447b823202c3c028ae1', 'request_id': 'req-01e03d49-5dfd-4e99-b397-08ef775c2597'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:13.926623 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-01e03d49-5dfd-4e99-b397-08ef775c2597 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '338b138d409f4447b823202c3c028ae1', 'request_id': 'req-01e03d49-5dfd-4e99-b397-08ef775c2597'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:13.940025 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-01e03d49-5dfd-4e99-b397-08ef775c2597 admin admin] The token cache is being invalidate because project 338b138d409f4447b823202c3c028ae1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:40:13.940313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-01e03d49-5dfd-4e99-b397-08ef775c2597 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:13.940936 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-01e03d49-5dfd-4e99-b397-08ef775c2597 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:13.942513 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1156/2281] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:40:13 2022] DELETE /identity/v3/projects/338b138d409f4447b823202c3c028ae1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:40:13.947580 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88c80f-8da2-4443-bb9f-c547bed2216a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:13.948081 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88c80f-8da2-4443-bb9f-c547bed2216a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:13.948285 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88c80f-8da2-4443-bb9f-c547bed2216a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:13.969512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1b88c80f-8da2-4443-bb9f-c547bed2216a None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:13.970876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1b88c80f-8da2-4443-bb9f-c547bed2216a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:14.041344 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1126/2282] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:14.050624 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1732947-d682-42e5-a736-194e71c381a1 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:14.051096 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1732947-d682-42e5-a736-194e71c381a1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:14.051447 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1732947-d682-42e5-a736-194e71c381a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:14.077863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-c1732947-d682-42e5-a736-194e71c381a1 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:14.077863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c1732947-d682-42e5-a736-194e71c381a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:14.085902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1157/2283] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:14.090877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b88c80f-8da2-4443-bb9f-c547bed2216a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:14.107312 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b88c80f-8da2-4443-bb9f-c547bed2216a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:14.145106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45830b69-d096-4fb8-8270-77e207e1e5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:14.146589 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45830b69-d096-4fb8-8270-77e207e1e5dd admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:14.147715 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45830b69-d096-4fb8-8270-77e207e1e5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:14.148718 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45830b69-d096-4fb8-8270-77e207e1e5dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:14.150764 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45830b69-d096-4fb8-8270-77e207e1e5dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:40:14.151165 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45830b69-d096-4fb8-8270-77e207e1e5dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:14.154850 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45830b69-d096-4fb8-8270-77e207e1e5dd admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:14.177247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1127/2284] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:40:14.178415 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1732947-d682-42e5-a736-194e71c381a1 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:14.196897 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1732947-d682-42e5-a736-194e71c381a1 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:14.202943 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12bce3b-1c8f-42a2-96e8-c069d25b0bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:14.203257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12bce3b-1c8f-42a2-96e8-c069d25b0bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:14.203556 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12bce3b-1c8f-42a2-96e8-c069d25b0bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:14.203841 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12bce3b-1c8f-42a2-96e8-c069d25b0bd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:14.204559 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12bce3b-1c8f-42a2-96e8-c069d25b0bd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:14.206676 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12bce3b-1c8f-42a2-96e8-c069d25b0bd6 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:14.252630 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1158/2285] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:40:14 2022] POST /identity/v3/projects => generated 399 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:40:14.253385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45830b69-d096-4fb8-8270-77e207e1e5dd admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:14.265009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45830b69-d096-4fb8-8270-77e207e1e5dd admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:14.326435 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda60ad5-4e92-476f-a027-65f18b3a14ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:14.327109 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda60ad5-4e92-476f-a027-65f18b3a14ef admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:14.327943 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda60ad5-4e92-476f-a027-65f18b3a14ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:14.327943 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda60ad5-4e92-476f-a027-65f18b3a14ef admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:14.328358 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda60ad5-4e92-476f-a027-65f18b3a14ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:14.330940 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda60ad5-4e92-476f-a027-65f18b3a14ef admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:14.354811 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1128/2286] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:40:14 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:40:14.356057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12bce3b-1c8f-42a2-96e8-c069d25b0bd6 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:14.366921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12bce3b-1c8f-42a2-96e8-c069d25b0bd6 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:14.373268 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73723204-4302-4481-9c15-6cffe249bdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:14.374023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73723204-4302-4481-9c15-6cffe249bdb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:14.374305 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73723204-4302-4481-9c15-6cffe249bdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:14.374588 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73723204-4302-4481-9c15-6cffe249bdb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:14.375409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73723204-4302-4481-9c15-6cffe249bdb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:14.377037 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73723204-4302-4481-9c15-6cffe249bdb4 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:14.394408 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1159/2287] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:40:14.398220 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda60ad5-4e92-476f-a027-65f18b3a14ef admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:14.410987 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda60ad5-4e92-476f-a027-65f18b3a14ef admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:14.417460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1656a-f29f-4f8d-8aa5-b9d92d16c4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:14.418200 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1656a-f29f-4f8d-8aa5-b9d92d16c4ea admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:14.418496 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1656a-f29f-4f8d-8aa5-b9d92d16c4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:14.418853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1656a-f29f-4f8d-8aa5-b9d92d16c4ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:14.419874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1656a-f29f-4f8d-8aa5-b9d92d16c4ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:14.421530 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1656a-f29f-4f8d-8aa5-b9d92d16c4ea admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:14.439529 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1129/2288] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:40:14.439639 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73723204-4302-4481-9c15-6cffe249bdb4 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:14.448864 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73723204-4302-4481-9c15-6cffe249bdb4 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:14.455260 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377b965-0a1c-4fd5-b8e1-69b63215ae8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:14.456168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377b965-0a1c-4fd5-b8e1-69b63215ae8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:14.456454 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377b965-0a1c-4fd5-b8e1-69b63215ae8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:14.456730 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377b965-0a1c-4fd5-b8e1-69b63215ae8d admin admin] PATH_INFO: `/v3/projects/56654097cda4421fbf67397b531551fc/users/9a5daabd12014c4f89184dddd57cec79/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:14.473286 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0377b965-0a1c-4fd5-b8e1-69b63215ae8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=56654097cda4421fbf67397b531551fc, user_id=9a5daabd12014c4f89184dddd57cec79, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:14.476049 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0377b965-0a1c-4fd5-b8e1-69b63215ae8d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:14.500761 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1160/2289] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:40:14 2022] PUT /identity/v3/projects/56654097cda4421fbf67397b531551fc/users/9a5daabd12014c4f89184dddd57cec79/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:40:14.502550 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0003da7-68b4-437c-8404-fcca373bc28b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:14.502924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0003da7-68b4-437c-8404-fcca373bc28b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:14.503212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0003da7-68b4-437c-8404-fcca373bc28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:14.531626 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b0003da7-68b4-437c-8404-fcca373bc28b None None] MFA Rules not processed for user `9a5daabd12014c4f89184dddd57cec79`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:14.532873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b0003da7-68b4-437c-8404-fcca373bc28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:14.585024 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:14.646884 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1130/2290] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:14.662404 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377b965-0a1c-4fd5-b8e1-69b63215ae8d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:14.664161 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0377b965-0a1c-4fd5-b8e1-69b63215ae8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:14.741614 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377b965-0a1c-4fd5-b8e1-69b63215ae8d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:14.753803 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea1b5d-f183-4339-83e6-d5cc13d4068d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:14.754561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea1b5d-f183-4339-83e6-d5cc13d4068d service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:14.754967 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea1b5d-f183-4339-83e6-d5cc13d4068d service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:14.755318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea1b5d-f183-4339-83e6-d5cc13d4068d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:14.764723 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea1b5d-f183-4339-83e6-d5cc13d4068d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:14.766353 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea1b5d-f183-4339-83e6-d5cc13d4068d service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:14.777528 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1161/2291] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:40:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:40:17.948448 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916829c-6ec2-40bf-9ec7-2f2911ecfba9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:17.949026 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916829c-6ec2-40bf-9ec7-2f2911ecfba9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:17.949026 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916829c-6ec2-40bf-9ec7-2f2911ecfba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:17.997322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5916829c-6ec2-40bf-9ec7-2f2911ecfba9 None None] MFA Rules not processed for user `32e6ace174a840f78c8d570719c3f631`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:17.998578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5916829c-6ec2-40bf-9ec7-2f2911ecfba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:18.074804 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1131/2292] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:18.089533 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea1b5d-f183-4339-83e6-d5cc13d4068d service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:18.091095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-40ea1b5d-f183-4339-83e6-d5cc13d4068d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:18.118460 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:18.172955 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea1b5d-f183-4339-83e6-d5cc13d4068d service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:18.178321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e5dc00-eddf-4838-a1e5-b73e3d406e04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:18.179002 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5dc00-eddf-4838-a1e5-b73e3d406e04 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:18.179284 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5dc00-eddf-4838-a1e5-b73e3d406e04 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:18.179555 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5dc00-eddf-4838-a1e5-b73e3d406e04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:18.188028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e5dc00-eddf-4838-a1e5-b73e3d406e04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:18.189880 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e5dc00-eddf-4838-a1e5-b73e3d406e04 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:18.198896 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1162/2293] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:40:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:40:19.844650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d40548-a24a-414a-9183-a85d06a084a3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:19.845124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d40548-a24a-414a-9183-a85d06a084a3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:19.845288 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d40548-a24a-414a-9183-a85d06a084a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:19.867781 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e2d40548-a24a-414a-9183-a85d06a084a3 None None] MFA Rules not processed for user `32e6ace174a840f78c8d570719c3f631`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:19.869897 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e2d40548-a24a-414a-9183-a85d06a084a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:19.890424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1132/2294] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:20.019704 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e18b4a-a763-421d-8804-e16e9f1d8cbc None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:20.020254 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e18b4a-a763-421d-8804-e16e9f1d8cbc None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:20.020254 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e18b4a-a763-421d-8804-e16e9f1d8cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:20.043936 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-39e18b4a-a763-421d-8804-e16e9f1d8cbc None None] MFA Rules not processed for user `ab5d06b0a3324dab903f485342590db9`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:20.046566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-39e18b4a-a763-421d-8804-e16e9f1d8cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:20.165738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1163/2295] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:40:29.860820 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=43 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:30.155165 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=49 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:43.121537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d40548-a24a-414a-9183-a85d06a084a3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:43.187024 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d40548-a24a-414a-9183-a85d06a084a3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:43.192388 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48658677-dc46-4657-9370-6edafb1723d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:43.193365 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48658677-dc46-4657-9370-6edafb1723d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:43.193647 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48658677-dc46-4657-9370-6edafb1723d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:43.193920 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48658677-dc46-4657-9370-6edafb1723d4 admin admin] PATH_INFO: `/v3/users/9a5daabd12014c4f89184dddd57cec79` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:43.195188 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48658677-dc46-4657-9370-6edafb1723d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a5daabd12014c4f89184dddd57cec79)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:43.197094 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48658677-dc46-4657-9370-6edafb1723d4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:43.245787 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-48658677-dc46-4657-9370-6edafb1723d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a5daabd12014c4f89184dddd57cec79'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:43.253145 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-48658677-dc46-4657-9370-6edafb1723d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a5daabd12014c4f89184dddd57cec79'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:43.260251 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-48658677-dc46-4657-9370-6edafb1723d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a5daabd12014c4f89184dddd57cec79'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:43.282930 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-48658677-dc46-4657-9370-6edafb1723d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a5daabd12014c4f89184dddd57cec79'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:43.283571 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-48658677-dc46-4657-9370-6edafb1723d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a5daabd12014c4f89184dddd57cec79'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:43.289588 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1133/2296] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:40:43 2022] DELETE /identity/v3/users/9a5daabd12014c4f89184dddd57cec79 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:40:43.294046 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09c8bc-538f-440b-9a0b-e82a2e59370d None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:43.294542 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09c8bc-538f-440b-9a0b-e82a2e59370d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:43.294755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09c8bc-538f-440b-9a0b-e82a2e59370d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:43.313745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-cf09c8bc-538f-440b-9a0b-e82a2e59370d None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:43.315270 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cf09c8bc-538f-440b-9a0b-e82a2e59370d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:43.386613 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1164/2297] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:43.396827 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48658677-dc46-4657-9370-6edafb1723d4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:43.398329 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-48658677-dc46-4657-9370-6edafb1723d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:43.458142 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48658677-dc46-4657-9370-6edafb1723d4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:43.462112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628d0f1-848a-41a4-a22c-4a9f6817781b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:43.462667 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628d0f1-848a-41a4-a22c-4a9f6817781b service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:43.463051 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628d0f1-848a-41a4-a22c-4a9f6817781b service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:43.463292 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628d0f1-848a-41a4-a22c-4a9f6817781b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:43.473672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628d0f1-848a-41a4-a22c-4a9f6817781b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:43.475170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628d0f1-848a-41a4-a22c-4a9f6817781b service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:43.486278 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1134/2298] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:40:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:40:43.749290 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09c8bc-538f-440b-9a0b-e82a2e59370d None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:43.751229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cf09c8bc-538f-440b-9a0b-e82a2e59370d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:43.763157 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09c8bc-538f-440b-9a0b-e82a2e59370d None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:43.768390 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c84bc4-bcb2-479a-961d-d0f019f45d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:43.769783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c84bc4-bcb2-479a-961d-d0f019f45d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:43.770094 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c84bc4-bcb2-479a-961d-d0f019f45d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:43.770346 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c84bc4-bcb2-479a-961d-d0f019f45d40 admin admin] PATH_INFO: `/v3/projects/56654097cda4421fbf67397b531551fc` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:43.771637 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c84bc4-bcb2-479a-961d-d0f019f45d40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56654097cda4421fbf67397b531551fc)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:43.774036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c84bc4-bcb2-479a-961d-d0f019f45d40 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:43.824951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-33c84bc4-bcb2-479a-961d-d0f019f45d40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56654097cda4421fbf67397b531551fc', 'request_id': 'req-33c84bc4-bcb2-479a-961d-d0f019f45d40'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:43.831309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-33c84bc4-bcb2-479a-961d-d0f019f45d40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56654097cda4421fbf67397b531551fc', 'request_id': 'req-33c84bc4-bcb2-479a-961d-d0f019f45d40'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:43.839275 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-33c84bc4-bcb2-479a-961d-d0f019f45d40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56654097cda4421fbf67397b531551fc', 'request_id': 'req-33c84bc4-bcb2-479a-961d-d0f019f45d40'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:43.855462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-33c84bc4-bcb2-479a-961d-d0f019f45d40 admin admin] The token cache is being invalidate because project 56654097cda4421fbf67397b531551fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:40:43.856140 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-33c84bc4-bcb2-479a-961d-d0f019f45d40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:43.856414 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-33c84bc4-bcb2-479a-961d-d0f019f45d40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:43.858253 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1165/2299] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:40:43 2022] DELETE /identity/v3/projects/56654097cda4421fbf67397b531551fc => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:40:47.340634 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628d0f1-848a-41a4-a22c-4a9f6817781b service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:47.342027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5628d0f1-848a-41a4-a22c-4a9f6817781b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:47.366169 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:47.420858 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628d0f1-848a-41a4-a22c-4a9f6817781b service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:47.476251 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eeb38b-4487-4fcf-9b9a-267ca72c5176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:47.477135 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eeb38b-4487-4fcf-9b9a-267ca72c5176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:47.477641 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eeb38b-4487-4fcf-9b9a-267ca72c5176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:47.477752 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eeb38b-4487-4fcf-9b9a-267ca72c5176 admin admin] PATH_INFO: `/v3/users/3c158f9c73ed4aa78d6d7916aa189607` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:47.478904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eeb38b-4487-4fcf-9b9a-267ca72c5176 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c158f9c73ed4aa78d6d7916aa189607)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:47.480789 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eeb38b-4487-4fcf-9b9a-267ca72c5176 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:47.524750 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-05eeb38b-4487-4fcf-9b9a-267ca72c5176 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c158f9c73ed4aa78d6d7916aa189607'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:47.532886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-05eeb38b-4487-4fcf-9b9a-267ca72c5176 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c158f9c73ed4aa78d6d7916aa189607'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:47.540606 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-05eeb38b-4487-4fcf-9b9a-267ca72c5176 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c158f9c73ed4aa78d6d7916aa189607'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:47.553766 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-05eeb38b-4487-4fcf-9b9a-267ca72c5176 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c158f9c73ed4aa78d6d7916aa189607'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:47.554458 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-05eeb38b-4487-4fcf-9b9a-267ca72c5176 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c158f9c73ed4aa78d6d7916aa189607'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:47.560894 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1135/2300] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:40:47 2022] DELETE /identity/v3/users/3c158f9c73ed4aa78d6d7916aa189607 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:40:47.565629 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c81b1-1ea4-4349-a557-0b207436effe None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:47.566070 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c81b1-1ea4-4349-a557-0b207436effe None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:47.566194 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c81b1-1ea4-4349-a557-0b207436effe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:47.592965 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-ff9c81b1-1ea4-4349-a557-0b207436effe None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:47.594886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ff9c81b1-1ea4-4349-a557-0b207436effe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:47.675956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1166/2301] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:40:47.685353 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eeb38b-4487-4fcf-9b9a-267ca72c5176 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:47.686770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-05eeb38b-4487-4fcf-9b9a-267ca72c5176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:47.733533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:47.758885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c81b1-1ea4-4349-a557-0b207436effe None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:47.760593 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ff9c81b1-1ea4-4349-a557-0b207436effe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:47.773277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c81b1-1ea4-4349-a557-0b207436effe None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:47.778293 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f777f-bd5d-4ad1-a93a-f10c3b511b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:47.779322 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f777f-bd5d-4ad1-a93a-f10c3b511b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:47.779601 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f777f-bd5d-4ad1-a93a-f10c3b511b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:47.779908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f777f-bd5d-4ad1-a93a-f10c3b511b31 admin admin] PATH_INFO: `/v3/users/6f678d348f22411cb767f5afaeecfa50` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:47.781755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f777f-bd5d-4ad1-a93a-f10c3b511b31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f678d348f22411cb767f5afaeecfa50)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:47.783803 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f777f-bd5d-4ad1-a93a-f10c3b511b31 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:47.789555 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eeb38b-4487-4fcf-9b9a-267ca72c5176 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:47.793487 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1caa039-2164-456f-84b1-4e286398c5e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:47.794427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1caa039-2164-456f-84b1-4e286398c5e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:47.794739 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1caa039-2164-456f-84b1-4e286398c5e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:47.795027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1caa039-2164-456f-84b1-4e286398c5e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:47.801465 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:47.805848 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1caa039-2164-456f-84b1-4e286398c5e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:47.808757 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1caa039-2164-456f-84b1-4e286398c5e8 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:47.826768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1136/2302] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:40:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:40:47.854375 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e41f777f-bd5d-4ad1-a93a-f10c3b511b31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f678d348f22411cb767f5afaeecfa50'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:47.864067 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e41f777f-bd5d-4ad1-a93a-f10c3b511b31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f678d348f22411cb767f5afaeecfa50'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:47.864725 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e41f777f-bd5d-4ad1-a93a-f10c3b511b31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f678d348f22411cb767f5afaeecfa50'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:47.881190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e41f777f-bd5d-4ad1-a93a-f10c3b511b31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f678d348f22411cb767f5afaeecfa50'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:47.882043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e41f777f-bd5d-4ad1-a93a-f10c3b511b31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f678d348f22411cb767f5afaeecfa50'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:47.900455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1167/2303] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:40:47 2022] DELETE /identity/v3/users/6f678d348f22411cb767f5afaeecfa50 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:40:47.903696 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ff370-a2a5-4779-ab7e-35231ebb8731 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:47.904712 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ff370-a2a5-4779-ab7e-35231ebb8731 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:47.905179 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ff370-a2a5-4779-ab7e-35231ebb8731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:47.926484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-be5ff370-a2a5-4779-ab7e-35231ebb8731 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:47.927945 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-be5ff370-a2a5-4779-ab7e-35231ebb8731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:47.996895 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1137/2304] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:48.013596 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f777f-bd5d-4ad1-a93a-f10c3b511b31 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:48.019193 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e41f777f-bd5d-4ad1-a93a-f10c3b511b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:48.144357 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f777f-bd5d-4ad1-a93a-f10c3b511b31 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:48.148792 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f9cc4-0925-4623-b109-fb7997c0f6a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:48.150719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f9cc4-0925-4623-b109-fb7997c0f6a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:48.150719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f9cc4-0925-4623-b109-fb7997c0f6a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:48.150719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f9cc4-0925-4623-b109-fb7997c0f6a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:48.167549 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f9cc4-0925-4623-b109-fb7997c0f6a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:48.169089 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f9cc4-0925-4623-b109-fb7997c0f6a6 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:48.183254 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1168/2305] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:40:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:40:48.355010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ff370-a2a5-4779-ab7e-35231ebb8731 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:48.357343 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-be5ff370-a2a5-4779-ab7e-35231ebb8731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:48.372681 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ff370-a2a5-4779-ab7e-35231ebb8731 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:48.378237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25321b-183b-4432-b0d7-62a6f7fdff64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:48.379571 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25321b-183b-4432-b0d7-62a6f7fdff64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:48.379571 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25321b-183b-4432-b0d7-62a6f7fdff64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:48.379752 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25321b-183b-4432-b0d7-62a6f7fdff64 admin admin] PATH_INFO: `/v3/projects/2ab54c8c8832499087505dcec487852d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:48.381419 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd25321b-183b-4432-b0d7-62a6f7fdff64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ab54c8c8832499087505dcec487852d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:48.383895 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd25321b-183b-4432-b0d7-62a6f7fdff64 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:48.482394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bd25321b-183b-4432-b0d7-62a6f7fdff64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ab54c8c8832499087505dcec487852d', 'request_id': 'req-bd25321b-183b-4432-b0d7-62a6f7fdff64'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:48.488920 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bd25321b-183b-4432-b0d7-62a6f7fdff64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ab54c8c8832499087505dcec487852d', 'request_id': 'req-bd25321b-183b-4432-b0d7-62a6f7fdff64'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:48.495992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bd25321b-183b-4432-b0d7-62a6f7fdff64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ab54c8c8832499087505dcec487852d', 'request_id': 'req-bd25321b-183b-4432-b0d7-62a6f7fdff64'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:48.509255 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bd25321b-183b-4432-b0d7-62a6f7fdff64 admin admin] The token cache is being invalidate because project 2ab54c8c8832499087505dcec487852d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:40:48.509801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bd25321b-183b-4432-b0d7-62a6f7fdff64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:48.510451 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bd25321b-183b-4432-b0d7-62a6f7fdff64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:48.512474 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1138/2306] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:40:48 2022] DELETE /identity/v3/projects/2ab54c8c8832499087505dcec487852d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:40:48.518278 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043cf90b-2db4-4108-af3c-dea332ea3547 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:48.518741 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043cf90b-2db4-4108-af3c-dea332ea3547 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:48.519088 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043cf90b-2db4-4108-af3c-dea332ea3547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:48.558847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-043cf90b-2db4-4108-af3c-dea332ea3547 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:48.560180 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-043cf90b-2db4-4108-af3c-dea332ea3547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:48.622733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1169/2307] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:48.631636 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73c4c0-260f-4d3e-acc0-e7c7a2cc4407 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:48.631929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73c4c0-260f-4d3e-acc0-e7c7a2cc4407 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:48.632214 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73c4c0-260f-4d3e-acc0-e7c7a2cc4407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:48.652239 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4c73c4c0-260f-4d3e-acc0-e7c7a2cc4407 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:48.653502 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4c73c4c0-260f-4d3e-acc0-e7c7a2cc4407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:48.662268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1139/2308] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:48.667249 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043cf90b-2db4-4108-af3c-dea332ea3547 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:48.679246 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043cf90b-2db4-4108-af3c-dea332ea3547 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:48.684047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc67e9-2edc-4935-9472-81ae49c17b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:48.688887 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc67e9-2edc-4935-9472-81ae49c17b0c admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:48.689179 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc67e9-2edc-4935-9472-81ae49c17b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:48.689460 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc67e9-2edc-4935-9472-81ae49c17b0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:48.689919 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73c4c0-260f-4d3e-acc0-e7c7a2cc4407 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:48.690232 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc67e9-2edc-4935-9472-81ae49c17b0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:40:48.690549 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc67e9-2edc-4935-9472-81ae49c17b0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:48.691412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4c73c4c0-260f-4d3e-acc0-e7c7a2cc4407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:48.692227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc67e9-2edc-4935-9472-81ae49c17b0c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:48.706906 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1170/2309] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:40:48.707387 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73c4c0-260f-4d3e-acc0-e7c7a2cc4407 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:48.711212 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc67e9-2edc-4935-9472-81ae49c17b0c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:48.713129 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f82f64-aa2a-426c-83a3-a0ec6cb49765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:48.714035 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f82f64-aa2a-426c-83a3-a0ec6cb49765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:48.714319 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f82f64-aa2a-426c-83a3-a0ec6cb49765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:48.714599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f82f64-aa2a-426c-83a3-a0ec6cb49765 admin admin] PATH_INFO: `/v3/users/1146f607aae246c39d69cdeca4c9cf48` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:48.715878 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f82f64-aa2a-426c-83a3-a0ec6cb49765 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1146f607aae246c39d69cdeca4c9cf48)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:48.717827 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f82f64-aa2a-426c-83a3-a0ec6cb49765 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:48.718542 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:48.723198 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc67e9-2edc-4935-9472-81ae49c17b0c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:48.731867 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b469b-e13a-4e6b-9b56-d2d69bbf7957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:48.732967 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b469b-e13a-4e6b-9b56-d2d69bbf7957 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:48.733865 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b469b-e13a-4e6b-9b56-d2d69bbf7957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:48.735260 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b469b-e13a-4e6b-9b56-d2d69bbf7957 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:48.736220 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b469b-e13a-4e6b-9b56-d2d69bbf7957 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:48.738568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b469b-e13a-4e6b-9b56-d2d69bbf7957 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:48.743987 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:48.772810 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1171/2310] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:40:48 2022] POST /identity/v3/projects => generated 361 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:40:48.777481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b469b-e13a-4e6b-9b56-d2d69bbf7957 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:48.783346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-88f82f64-aa2a-426c-83a3-a0ec6cb49765 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1146f607aae246c39d69cdeca4c9cf48'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:48.792295 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-88f82f64-aa2a-426c-83a3-a0ec6cb49765 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1146f607aae246c39d69cdeca4c9cf48'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:48.800371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-88f82f64-aa2a-426c-83a3-a0ec6cb49765 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1146f607aae246c39d69cdeca4c9cf48'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:48.816468 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-88f82f64-aa2a-426c-83a3-a0ec6cb49765 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1146f607aae246c39d69cdeca4c9cf48'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:48.817584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-88f82f64-aa2a-426c-83a3-a0ec6cb49765 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1146f607aae246c39d69cdeca4c9cf48'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:48.827698 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1140/2311] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:40:48 2022] DELETE /identity/v3/users/1146f607aae246c39d69cdeca4c9cf48 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:40:48.832588 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1003d9-c113-410b-a031-8c380f863da0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:48.832888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1003d9-c113-410b-a031-8c380f863da0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:48.833191 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1003d9-c113-410b-a031-8c380f863da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:48.853412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4e1003d9-c113-410b-a031-8c380f863da0 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:48.854687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4e1003d9-c113-410b-a031-8c380f863da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:48.864473 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b469b-e13a-4e6b-9b56-d2d69bbf7957 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:48.926873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1141/2312] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:40:48.936289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f0f7a-3cc0-4de6-99da-34cbe5cbca3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:48.936983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f0f7a-3cc0-4de6-99da-34cbe5cbca3d admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:48.937272 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f0f7a-3cc0-4de6-99da-34cbe5cbca3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:48.937591 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f0f7a-3cc0-4de6-99da-34cbe5cbca3d admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:48.938160 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1003d9-c113-410b-a031-8c380f863da0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:48.938559 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f0f7a-3cc0-4de6-99da-34cbe5cbca3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:48.939816 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4e1003d9-c113-410b-a031-8c380f863da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:48.940555 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f0f7a-3cc0-4de6-99da-34cbe5cbca3d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:48.964703 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1172/2313] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:40:48 2022] POST /identity/v3/users => generated 281 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:40:48.968657 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f0f7a-3cc0-4de6-99da-34cbe5cbca3d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:48.971242 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-489f0f7a-3cc0-4de6-99da-34cbe5cbca3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:48.984537 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f0f7a-3cc0-4de6-99da-34cbe5cbca3d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:48.989847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7eb6d-fd8a-411a-bdd0-b24405ea5f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:48.990672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7eb6d-fd8a-411a-bdd0-b24405ea5f80 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:48.991768 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7eb6d-fd8a-411a-bdd0-b24405ea5f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:48.991768 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7eb6d-fd8a-411a-bdd0-b24405ea5f80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:48.992317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d7eb6d-fd8a-411a-bdd0-b24405ea5f80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:48.995233 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d7eb6d-fd8a-411a-bdd0-b24405ea5f80 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:49.008107 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1173/2314] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:40:49.011851 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7eb6d-fd8a-411a-bdd0-b24405ea5f80 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:49.022304 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7eb6d-fd8a-411a-bdd0-b24405ea5f80 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:49.023023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1003d9-c113-410b-a031-8c380f863da0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:49.027138 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e1ddf-83b0-4e1b-91ad-0de06279a6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:49.027807 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e1ddf-83b0-4e1b-91ad-0de06279a6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:49.028078 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a04d7-fc07-4ae6-8fef-3cebf2ac3c7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:49.028215 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e1ddf-83b0-4e1b-91ad-0de06279a6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:49.029172 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a04d7-fc07-4ae6-8fef-3cebf2ac3c7c service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:49.029558 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a04d7-fc07-4ae6-8fef-3cebf2ac3c7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:49.029916 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e1ddf-83b0-4e1b-91ad-0de06279a6b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:49.030615 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e1ddf-83b0-4e1b-91ad-0de06279a6b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:49.032900 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e1ddf-83b0-4e1b-91ad-0de06279a6b9 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:49.034782 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a04d7-fc07-4ae6-8fef-3cebf2ac3c7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:49.046705 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1174/2315] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:40:49.048395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a04d7-fc07-4ae6-8fef-3cebf2ac3c7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:49.050440 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e1ddf-83b0-4e1b-91ad-0de06279a6b9 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:49.050604 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a04d7-fc07-4ae6-8fef-3cebf2ac3c7c service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:49.062492 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e1ddf-83b0-4e1b-91ad-0de06279a6b9 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:49.069446 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99939d-1489-483a-99c8-0d6ea2d17670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:49.070774 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99939d-1489-483a-99c8-0d6ea2d17670 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:49.071128 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99939d-1489-483a-99c8-0d6ea2d17670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:49.071480 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99939d-1489-483a-99c8-0d6ea2d17670 admin admin] PATH_INFO: `/v3/projects/ba7a43162de548d9ad567b13d58f8014/users/3c4f08e618ed44e7847c0da3a00a6ced/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:49.075797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1142/2316] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:40:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:40:49.093733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99939d-1489-483a-99c8-0d6ea2d17670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba7a43162de548d9ad567b13d58f8014, user_id=3c4f08e618ed44e7847c0da3a00a6ced, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:49.097917 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99939d-1489-483a-99c8-0d6ea2d17670 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:49.114248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1175/2317] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:40:49 2022] PUT /identity/v3/projects/ba7a43162de548d9ad567b13d58f8014/users/3c4f08e618ed44e7847c0da3a00a6ced/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:40:49.124544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c67ac5-02e2-4eb2-81ed-a31531f7d5c7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:49.125384 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c67ac5-02e2-4eb2-81ed-a31531f7d5c7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:49.125828 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c67ac5-02e2-4eb2-81ed-a31531f7d5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:49.156812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-32c67ac5-02e2-4eb2-81ed-a31531f7d5c7 None None] MFA Rules not processed for user `3c4f08e618ed44e7847c0da3a00a6ced`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:49.157984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-32c67ac5-02e2-4eb2-81ed-a31531f7d5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:49.189578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:49.250802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1143/2318] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:49 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:49.289825 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99939d-1489-483a-99c8-0d6ea2d17670 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:49.292658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5e99939d-1489-483a-99c8-0d6ea2d17670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:49.305835 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c67ac5-02e2-4eb2-81ed-a31531f7d5c7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:49.306706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-32c67ac5-02e2-4eb2-81ed-a31531f7d5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:49.341644 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:49.370118 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99939d-1489-483a-99c8-0d6ea2d17670 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:49.374733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23f9af-9b74-4406-b2f6-76660b41fb22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:49.376283 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23f9af-9b74-4406-b2f6-76660b41fb22 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:49.376405 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23f9af-9b74-4406-b2f6-76660b41fb22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:49.376808 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23f9af-9b74-4406-b2f6-76660b41fb22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:49.386019 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d23f9af-9b74-4406-b2f6-76660b41fb22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:49.387653 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d23f9af-9b74-4406-b2f6-76660b41fb22 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:49.398956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1176/2319] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:40:49 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:40:49.417041 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23f9af-9b74-4406-b2f6-76660b41fb22 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:49.427586 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c67ac5-02e2-4eb2-81ed-a31531f7d5c7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:49.433004 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23f9af-9b74-4406-b2f6-76660b41fb22 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:49.433843 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf89302-d1fb-491b-b14b-8a6c6b351ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:49.435156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf89302-d1fb-491b-b14b-8a6c6b351ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:49.435636 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf89302-d1fb-491b-b14b-8a6c6b351ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:49.436128 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf89302-d1fb-491b-b14b-8a6c6b351ae4 admin admin] PATH_INFO: `/v3/projects/60a94ab549b04c58b8341229fb716254` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:49.438349 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf89302-d1fb-491b-b14b-8a6c6b351ae4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60a94ab549b04c58b8341229fb716254)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:49.440292 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf89302-d1fb-491b-b14b-8a6c6b351ae4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:49.440407 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976438ca-9198-4063-8b6f-f324beb81371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:49.441365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976438ca-9198-4063-8b6f-f324beb81371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:49.441746 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976438ca-9198-4063-8b6f-f324beb81371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:49.442122 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976438ca-9198-4063-8b6f-f324beb81371 admin admin] PATH_INFO: `/v3/users/3c4f08e618ed44e7847c0da3a00a6ced` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:49.452011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976438ca-9198-4063-8b6f-f324beb81371 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c4f08e618ed44e7847c0da3a00a6ced)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:49.455681 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976438ca-9198-4063-8b6f-f324beb81371 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:49.519793 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2bf89302-d1fb-491b-b14b-8a6c6b351ae4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60a94ab549b04c58b8341229fb716254', 'request_id': 'req-2bf89302-d1fb-491b-b14b-8a6c6b351ae4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:49.526790 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2bf89302-d1fb-491b-b14b-8a6c6b351ae4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60a94ab549b04c58b8341229fb716254', 'request_id': 'req-2bf89302-d1fb-491b-b14b-8a6c6b351ae4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:49.534300 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2bf89302-d1fb-491b-b14b-8a6c6b351ae4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60a94ab549b04c58b8341229fb716254', 'request_id': 'req-2bf89302-d1fb-491b-b14b-8a6c6b351ae4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:49.542986 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-976438ca-9198-4063-8b6f-f324beb81371 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c4f08e618ed44e7847c0da3a00a6ced'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:49.556230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2bf89302-d1fb-491b-b14b-8a6c6b351ae4 admin admin] The token cache is being invalidate because project 60a94ab549b04c58b8341229fb716254 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:40:49.556551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2bf89302-d1fb-491b-b14b-8a6c6b351ae4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:49.557255 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2bf89302-d1fb-491b-b14b-8a6c6b351ae4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:49.560633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1144/2320] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:40:49 2022] DELETE /identity/v3/projects/60a94ab549b04c58b8341229fb716254 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:40:49.564295 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-976438ca-9198-4063-8b6f-f324beb81371 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c4f08e618ed44e7847c0da3a00a6ced'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:49.565131 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-976438ca-9198-4063-8b6f-f324beb81371 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c4f08e618ed44e7847c0da3a00a6ced'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:49.565646 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2938564-cb3d-40e4-8f23-3accb5f1538c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:49.565930 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2938564-cb3d-40e4-8f23-3accb5f1538c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:49.566217 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2938564-cb3d-40e4-8f23-3accb5f1538c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:49.580691 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-976438ca-9198-4063-8b6f-f324beb81371 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c4f08e618ed44e7847c0da3a00a6ced'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:49.581583 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-976438ca-9198-4063-8b6f-f324beb81371 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c4f08e618ed44e7847c0da3a00a6ced'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:49.586263 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f2938564-cb3d-40e4-8f23-3accb5f1538c None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:49.589207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f2938564-cb3d-40e4-8f23-3accb5f1538c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:49.598559 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1177/2321] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:40:49 2022] DELETE /identity/v3/users/3c4f08e618ed44e7847c0da3a00a6ced => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:40:49.600382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71700282-8f31-4b36-997c-97971b974b81 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:49.600507 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71700282-8f31-4b36-997c-97971b974b81 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:49.601204 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71700282-8f31-4b36-997c-97971b974b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:49.636538 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-71700282-8f31-4b36-997c-97971b974b81 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:49.639474 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-71700282-8f31-4b36-997c-97971b974b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:49.675688 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:49.694572 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1145/2322] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:40:49.695487 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2938564-cb3d-40e4-8f23-3accb5f1538c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:49.696806 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f2938564-cb3d-40e4-8f23-3accb5f1538c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:49.709803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2938564-cb3d-40e4-8f23-3accb5f1538c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:49.729212 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1178/2323] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:49.730738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d088914-ba71-4467-aae0-49bcd2c69513 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:49.731122 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d088914-ba71-4467-aae0-49bcd2c69513 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:49.731495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d088914-ba71-4467-aae0-49bcd2c69513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:49.762631 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6d088914-ba71-4467-aae0-49bcd2c69513 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:49.764756 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6d088914-ba71-4467-aae0-49bcd2c69513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:49.775682 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1179/2324] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:49.776907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d088914-ba71-4467-aae0-49bcd2c69513 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:49.778162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6d088914-ba71-4467-aae0-49bcd2c69513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:49.788997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e04c33-406d-48ef-991f-70fc4c3fcf78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:49.789876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e04c33-406d-48ef-991f-70fc4c3fcf78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:49.790166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e04c33-406d-48ef-991f-70fc4c3fcf78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:49.790450 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e04c33-406d-48ef-991f-70fc4c3fcf78 admin admin] PATH_INFO: `/v3/projects/3c38356bdf904142aaca69087953f480` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:49.791671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e04c33-406d-48ef-991f-70fc4c3fcf78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c38356bdf904142aaca69087953f480)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:49.793568 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e04c33-406d-48ef-991f-70fc4c3fcf78 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:49.828326 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:49.842927 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-30e04c33-406d-48ef-991f-70fc4c3fcf78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c38356bdf904142aaca69087953f480', 'request_id': 'req-30e04c33-406d-48ef-991f-70fc4c3fcf78'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:49.843388 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d088914-ba71-4467-aae0-49bcd2c69513 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:49.849409 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-30e04c33-406d-48ef-991f-70fc4c3fcf78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c38356bdf904142aaca69087953f480', 'request_id': 'req-30e04c33-406d-48ef-991f-70fc4c3fcf78'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:49.855839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-30e04c33-406d-48ef-991f-70fc4c3fcf78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c38356bdf904142aaca69087953f480', 'request_id': 'req-30e04c33-406d-48ef-991f-70fc4c3fcf78'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:49.873128 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-30e04c33-406d-48ef-991f-70fc4c3fcf78 admin admin] The token cache is being invalidate because project 3c38356bdf904142aaca69087953f480 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:40:49.873569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-30e04c33-406d-48ef-991f-70fc4c3fcf78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:49.874165 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-30e04c33-406d-48ef-991f-70fc4c3fcf78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:49.875997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1146/2325] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:40:49 2022] DELETE /identity/v3/projects/3c38356bdf904142aaca69087953f480 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:40:49.876906 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e04c33-406d-48ef-991f-70fc4c3fcf78 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:49.878315 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-30e04c33-406d-48ef-991f-70fc4c3fcf78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:49.914815 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988b6cc-3cb6-466d-8a16-10acbdbf4bb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:49.915938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988b6cc-3cb6-466d-8a16-10acbdbf4bb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:49.916287 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988b6cc-3cb6-466d-8a16-10acbdbf4bb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:49.916782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988b6cc-3cb6-466d-8a16-10acbdbf4bb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:49.923945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5988b6cc-3cb6-466d-8a16-10acbdbf4bb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:49.954291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e04c33-406d-48ef-991f-70fc4c3fcf78 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:49.960788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf27342-a36d-4cfe-8929-a2fb43b895ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:49.961614 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf27342-a36d-4cfe-8929-a2fb43b895ee admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:49.961996 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf27342-a36d-4cfe-8929-a2fb43b895ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:49.962371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf27342-a36d-4cfe-8929-a2fb43b895ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:49.963240 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf27342-a36d-4cfe-8929-a2fb43b895ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:40:49.963697 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf27342-a36d-4cfe-8929-a2fb43b895ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:49.965388 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf27342-a36d-4cfe-8929-a2fb43b895ee admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:49.978563 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1147/2326] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:40:49.983057 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09800eac-62be-4100-b0e8-3937bfff6185 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:49.983344 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09800eac-62be-4100-b0e8-3937bfff6185 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:49.983623 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09800eac-62be-4100-b0e8-3937bfff6185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:50.003623 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-09800eac-62be-4100-b0e8-3937bfff6185 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:50.004844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-09800eac-62be-4100-b0e8-3937bfff6185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:50.007835 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5988b6cc-3cb6-466d-8a16-10acbdbf4bb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:50.009291 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5988b6cc-3cb6-466d-8a16-10acbdbf4bb0 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:50.013058 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1148/2327] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:50.015497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09800eac-62be-4100-b0e8-3937bfff6185 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:50.022300 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1180/2328] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:40:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:40:50.024059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a003c37-aeeb-43ee-b465-ee2d64b068be None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:50.024694 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a003c37-aeeb-43ee-b465-ee2d64b068be None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:50.024694 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a003c37-aeeb-43ee-b465-ee2d64b068be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:50.029053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09800eac-62be-4100-b0e8-3937bfff6185 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:50.035973 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99194052-aeeb-44c6-a649-72a060d60459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:50.036861 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99194052-aeeb-44c6-a649-72a060d60459 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:50.037292 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99194052-aeeb-44c6-a649-72a060d60459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:50.037737 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99194052-aeeb-44c6-a649-72a060d60459 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:50.038582 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99194052-aeeb-44c6-a649-72a060d60459 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:50.041799 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99194052-aeeb-44c6-a649-72a060d60459 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:50.044461 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-7a003c37-aeeb-43ee-b465-ee2d64b068be None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:50.045689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7a003c37-aeeb-43ee-b465-ee2d64b068be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:50.054769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1181/2329] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:50.059527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:50.059711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a003c37-aeeb-43ee-b465-ee2d64b068be None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:50.073058 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a003c37-aeeb-43ee-b465-ee2d64b068be None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:50.079343 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1149/2330] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:40:50 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:40:50.081210 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f9b02-e046-4dda-892a-6aa5546d4884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:50.081992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f9b02-e046-4dda-892a-6aa5546d4884 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:50.082457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f9b02-e046-4dda-892a-6aa5546d4884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:50.083637 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99194052-aeeb-44c6-a649-72a060d60459 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:50.083916 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f9b02-e046-4dda-892a-6aa5546d4884 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:50.084762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21f9b02-e046-4dda-892a-6aa5546d4884 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:40:50.085674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21f9b02-e046-4dda-892a-6aa5546d4884 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:50.089537 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21f9b02-e046-4dda-892a-6aa5546d4884 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:50.113274 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1182/2331] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:40:50.114598 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f9b02-e046-4dda-892a-6aa5546d4884 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:50.137653 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f9b02-e046-4dda-892a-6aa5546d4884 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:50.169276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99194052-aeeb-44c6-a649-72a060d60459 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:50.176292 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3a9ac-9aeb-4981-9710-5ac3ef11485c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:50.177218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3a9ac-9aeb-4981-9710-5ac3ef11485c admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:50.177654 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3a9ac-9aeb-4981-9710-5ac3ef11485c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:50.178005 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3a9ac-9aeb-4981-9710-5ac3ef11485c admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:50.178840 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3a9ac-9aeb-4981-9710-5ac3ef11485c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:50.182940 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3a9ac-9aeb-4981-9710-5ac3ef11485c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:50.220423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1150/2332] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:40:50 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:40:50.224353 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3a9ac-9aeb-4981-9710-5ac3ef11485c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:50.235351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bec7a0-fa45-44e0-83c4-fb2b65467163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:50.236318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3a9ac-9aeb-4981-9710-5ac3ef11485c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:50.236598 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bec7a0-fa45-44e0-83c4-fb2b65467163 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:50.236888 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bec7a0-fa45-44e0-83c4-fb2b65467163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:50.241899 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5599b9-407a-43cb-9712-19a9c5c03422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:50.242402 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5599b9-407a-43cb-9712-19a9c5c03422 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:50.242720 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5599b9-407a-43cb-9712-19a9c5c03422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:50.242929 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bec7a0-fa45-44e0-83c4-fb2b65467163 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:50.243040 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5599b9-407a-43cb-9712-19a9c5c03422 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:50.243847 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5599b9-407a-43cb-9712-19a9c5c03422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:50.244522 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bec7a0-fa45-44e0-83c4-fb2b65467163 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:50.245611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5599b9-407a-43cb-9712-19a9c5c03422 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:50.247332 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bec7a0-fa45-44e0-83c4-fb2b65467163 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:50.256978 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1151/2333] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:40:50.260776 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5599b9-407a-43cb-9712-19a9c5c03422 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:50.272778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5599b9-407a-43cb-9712-19a9c5c03422 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:50.280123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1183/2334] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:40:50 2022] POST /identity/v3/projects => generated 373 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:40:50.280307 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6242b2e-8081-443e-afab-89767430f286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:50.280968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6242b2e-8081-443e-afab-89767430f286 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:50.281842 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6242b2e-8081-443e-afab-89767430f286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:50.282129 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6242b2e-8081-443e-afab-89767430f286 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:50.282908 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6242b2e-8081-443e-afab-89767430f286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:50.284705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6242b2e-8081-443e-afab-89767430f286 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:50.284826 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bec7a0-fa45-44e0-83c4-fb2b65467163 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:50.298155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:50.299033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1152/2335] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:40:50.303074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6242b2e-8081-443e-afab-89767430f286 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:50.314035 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6242b2e-8081-443e-afab-89767430f286 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:50.365770 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bec7a0-fa45-44e0-83c4-fb2b65467163 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:50.371394 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a58bd-0705-46fa-bd4a-363208cea401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:50.372560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a58bd-0705-46fa-bd4a-363208cea401 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:50.372857 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a58bd-0705-46fa-bd4a-363208cea401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:50.373139 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a58bd-0705-46fa-bd4a-363208cea401 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:50.374730 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a58bd-0705-46fa-bd4a-363208cea401 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:50.377240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a58bd-0705-46fa-bd4a-363208cea401 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:50.389323 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc04a9-070c-4815-a9e0-6a34b03fe688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:50.390994 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc04a9-070c-4815-a9e0-6a34b03fe688 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:50.391503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc04a9-070c-4815-a9e0-6a34b03fe688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:50.391969 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc04a9-070c-4815-a9e0-6a34b03fe688 admin admin] PATH_INFO: `/v3/projects/30ba4559e7f8482e9c28b9735efc770d/users/556af1d5d5f34585bc5dad930041b0f6/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:50.405010 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1184/2336] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:40:50 2022] POST /identity/v3/users => generated 287 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:40:50.407620 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a58bd-0705-46fa-bd4a-363208cea401 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:50.410319 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc04a9-070c-4815-a9e0-6a34b03fe688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30ba4559e7f8482e9c28b9735efc770d, user_id=556af1d5d5f34585bc5dad930041b0f6, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:50.411322 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-484a58bd-0705-46fa-bd4a-363208cea401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:50.415181 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc04a9-070c-4815-a9e0-6a34b03fe688 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:50.434702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a58bd-0705-46fa-bd4a-363208cea401 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:50.445683 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1153/2337] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:40:50 2022] PUT /identity/v3/projects/30ba4559e7f8482e9c28b9735efc770d/users/556af1d5d5f34585bc5dad930041b0f6/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:40:50.448794 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc04a9-070c-4815-a9e0-6a34b03fe688 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:50.512463 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba9693-6487-4d9b-9ed2-2aee86d6c82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:50.513496 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba9693-6487-4d9b-9ed2-2aee86d6c82a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:50.513874 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba9693-6487-4d9b-9ed2-2aee86d6c82a admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:50.514221 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba9693-6487-4d9b-9ed2-2aee86d6c82a admin admin] PATH_INFO: `/v3/projects/ba7a43162de548d9ad567b13d58f8014` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:50.516455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ba9693-6487-4d9b-9ed2-2aee86d6c82a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba7a43162de548d9ad567b13d58f8014)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:50.518437 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ba9693-6487-4d9b-9ed2-2aee86d6c82a admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:50.541606 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:50.563914 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc04a9-070c-4815-a9e0-6a34b03fe688 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:50.570019 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0846b4-4af4-4db7-80f8-96233ebf1535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:50.570863 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0846b4-4af4-4db7-80f8-96233ebf1535 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:50.571221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0846b4-4af4-4db7-80f8-96233ebf1535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:50.571706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0846b4-4af4-4db7-80f8-96233ebf1535 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:50.572699 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0846b4-4af4-4db7-80f8-96233ebf1535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:50.583260 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0846b4-4af4-4db7-80f8-96233ebf1535 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:50.585264 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-82ba9693-6487-4d9b-9ed2-2aee86d6c82a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba7a43162de548d9ad567b13d58f8014', 'request_id': 'req-82ba9693-6487-4d9b-9ed2-2aee86d6c82a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:50.590677 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1154/2338] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:40:50.592095 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256817b-bc1c-4333-8a0b-f4b6d11a08d5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:50.592393 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256817b-bc1c-4333-8a0b-f4b6d11a08d5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:50.592508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-82ba9693-6487-4d9b-9ed2-2aee86d6c82a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba7a43162de548d9ad567b13d58f8014', 'request_id': 'req-82ba9693-6487-4d9b-9ed2-2aee86d6c82a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:50.592709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256817b-bc1c-4333-8a0b-f4b6d11a08d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:50.603602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-82ba9693-6487-4d9b-9ed2-2aee86d6c82a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba7a43162de548d9ad567b13d58f8014', 'request_id': 'req-82ba9693-6487-4d9b-9ed2-2aee86d6c82a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:50.623373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1256817b-bc1c-4333-8a0b-f4b6d11a08d5 None None] MFA Rules not processed for user `556af1d5d5f34585bc5dad930041b0f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:50.625932 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1256817b-bc1c-4333-8a0b-f4b6d11a08d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:50.628108 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-82ba9693-6487-4d9b-9ed2-2aee86d6c82a admin admin] The token cache is being invalidate because project ba7a43162de548d9ad567b13d58f8014 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:40:50.628412 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-82ba9693-6487-4d9b-9ed2-2aee86d6c82a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:50.629081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-82ba9693-6487-4d9b-9ed2-2aee86d6c82a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:50.632063 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1185/2339] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:40:50 2022] DELETE /identity/v3/projects/ba7a43162de548d9ad567b13d58f8014 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:40:50.642022 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba9693-6487-4d9b-9ed2-2aee86d6c82a admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:50.645009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-82ba9693-6487-4d9b-9ed2-2aee86d6c82a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:50.694409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:50.716384 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1155/2340] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:50.718228 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a764fb-e569-4259-988a-b43f0588f6c6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:50.718228 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a764fb-e569-4259-988a-b43f0588f6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:50.718418 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a764fb-e569-4259-988a-b43f0588f6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:50.737623 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a8a764fb-e569-4259-988a-b43f0588f6c6 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:50.738903 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a8a764fb-e569-4259-988a-b43f0588f6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:50.740473 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba9693-6487-4d9b-9ed2-2aee86d6c82a admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:50.748245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1156/2341] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:50.749177 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a764fb-e569-4259-988a-b43f0588f6c6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:50.751197 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a8a764fb-e569-4259-988a-b43f0588f6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:50.755188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c3940-ca71-4cf7-b017-a4c51afcb906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:50.755936 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c3940-ca71-4cf7-b017-a4c51afcb906 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:50.756297 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c3940-ca71-4cf7-b017-a4c51afcb906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:50.756643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c3940-ca71-4cf7-b017-a4c51afcb906 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:50.757880 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c3940-ca71-4cf7-b017-a4c51afcb906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:50.759723 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c3940-ca71-4cf7-b017-a4c51afcb906 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:50.777312 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1186/2342] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:40:50.777312 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c63105-826a-4a84-93b4-9138077cc7bb None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:50.777312 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c63105-826a-4a84-93b4-9138077cc7bb None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:50.777312 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c63105-826a-4a84-93b4-9138077cc7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:50.807744 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-29c63105-826a-4a84-93b4-9138077cc7bb None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:50.813067 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-29c63105-826a-4a84-93b4-9138077cc7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:50.823836 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1187/2343] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:50.827586 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c63105-826a-4a84-93b4-9138077cc7bb None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:50.833237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a764fb-e569-4259-988a-b43f0588f6c6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:50.838552 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7ad5b-0bd7-4372-8605-a7e23583c66c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:50.839246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7ad5b-0bd7-4372-8605-a7e23583c66c service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:50.839422 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c63105-826a-4a84-93b4-9138077cc7bb None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:50.839524 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7ad5b-0bd7-4372-8605-a7e23583c66c service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:50.840476 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7ad5b-0bd7-4372-8605-a7e23583c66c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:50.845209 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d487fb-39ed-4f16-b2df-73c681e5b252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:50.846248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d487fb-39ed-4f16-b2df-73c681e5b252 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:50.846625 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d487fb-39ed-4f16-b2df-73c681e5b252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:50.847028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d487fb-39ed-4f16-b2df-73c681e5b252 admin admin] PATH_INFO: `/v3/projects/7ea9415341504da0a28fa1ac2deba1ca/users/0ccc5f10cf584b8ca5bdc158a02c2ce1/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:50.850824 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7ad5b-0bd7-4372-8605-a7e23583c66c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:50.852258 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7ad5b-0bd7-4372-8605-a7e23583c66c service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:50.862519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1157/2344] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:40:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:40:50.863255 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7ad5b-0bd7-4372-8605-a7e23583c66c service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:50.871144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d487fb-39ed-4f16-b2df-73c681e5b252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ea9415341504da0a28fa1ac2deba1ca, user_id=0ccc5f10cf584b8ca5bdc158a02c2ce1, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:50.873765 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7ad5b-0bd7-4372-8605-a7e23583c66c service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:50.875350 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d487fb-39ed-4f16-b2df-73c681e5b252 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:50.879574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e2108-94f1-45fc-a06c-d62d4be5f765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:50.880245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e2108-94f1-45fc-a06c-d62d4be5f765 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:50.881095 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e2108-94f1-45fc-a06c-d62d4be5f765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:50.881633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e2108-94f1-45fc-a06c-d62d4be5f765 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:50.882706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e2108-94f1-45fc-a06c-d62d4be5f765 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:40:50.882993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e2108-94f1-45fc-a06c-d62d4be5f765 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:50.884754 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e2108-94f1-45fc-a06c-d62d4be5f765 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:50.900760 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1158/2345] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:40:50.900760 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e2108-94f1-45fc-a06c-d62d4be5f765 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:50.900760 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0b3e2108-94f1-45fc-a06c-d62d4be5f765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:50.902015 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1188/2346] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:40:50 2022] PUT /identity/v3/projects/7ea9415341504da0a28fa1ac2deba1ca/users/0ccc5f10cf584b8ca5bdc158a02c2ce1/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:40:50.902981 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d487fb-39ed-4f16-b2df-73c681e5b252 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:50.918174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e2108-94f1-45fc-a06c-d62d4be5f765 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:50.980232 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d487fb-39ed-4f16-b2df-73c681e5b252 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:50.980602 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:50.987740 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9534dc-0578-40be-8b2f-5751ab374989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:50.988394 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9534dc-0578-40be-8b2f-5751ab374989 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:50.988680 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9534dc-0578-40be-8b2f-5751ab374989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:50.989090 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9534dc-0578-40be-8b2f-5751ab374989 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:50.989713 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9534dc-0578-40be-8b2f-5751ab374989 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:50.989841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7c661-36f7-48bf-87cc-a61fa3e76e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:50.990829 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7c661-36f7-48bf-87cc-a61fa3e76e73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:50.991175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7c661-36f7-48bf-87cc-a61fa3e76e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:50.991622 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7c661-36f7-48bf-87cc-a61fa3e76e73 admin admin] PATH_INFO: `/v3/users/556af1d5d5f34585bc5dad930041b0f6` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:50.992326 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9534dc-0578-40be-8b2f-5751ab374989 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:50.993121 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b7c661-36f7-48bf-87cc-a61fa3e76e73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=556af1d5d5f34585bc5dad930041b0f6)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:50.995081 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b7c661-36f7-48bf-87cc-a61fa3e76e73 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:51.041866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1189/2347] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:40:50 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:40:51.043377 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479932d0-0541-4465-8b30-9fbd75191e78 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:51.043693 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479932d0-0541-4465-8b30-9fbd75191e78 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:51.043944 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479932d0-0541-4465-8b30-9fbd75191e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:51.067537 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e5b7c661-36f7-48bf-87cc-a61fa3e76e73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '556af1d5d5f34585bc5dad930041b0f6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:51.074342 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-479932d0-0541-4465-8b30-9fbd75191e78 None None] MFA Rules not processed for user `0ccc5f10cf584b8ca5bdc158a02c2ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:51.075660 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-479932d0-0541-4465-8b30-9fbd75191e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:51.076437 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e5b7c661-36f7-48bf-87cc-a61fa3e76e73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '556af1d5d5f34585bc5dad930041b0f6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:51.087457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e5b7c661-36f7-48bf-87cc-a61fa3e76e73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '556af1d5d5f34585bc5dad930041b0f6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:51.105954 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e5b7c661-36f7-48bf-87cc-a61fa3e76e73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '556af1d5d5f34585bc5dad930041b0f6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:51.109015 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e5b7c661-36f7-48bf-87cc-a61fa3e76e73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '556af1d5d5f34585bc5dad930041b0f6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:51.121055 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1159/2348] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:40:50 2022] DELETE /identity/v3/users/556af1d5d5f34585bc5dad930041b0f6 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:40:51.122074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7c661-36f7-48bf-87cc-a61fa3e76e73 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:51.123657 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e5b7c661-36f7-48bf-87cc-a61fa3e76e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:51.164410 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1190/2349] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:51.170681 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0808606-13eb-406b-acde-d9e52e33711f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:51.171145 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0808606-13eb-406b-acde-d9e52e33711f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:51.171693 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0808606-13eb-406b-acde-d9e52e33711f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:51.205754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-c0808606-13eb-406b-acde-d9e52e33711f None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:51.206726 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7c661-36f7-48bf-87cc-a61fa3e76e73 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:51.208333 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c0808606-13eb-406b-acde-d9e52e33711f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:51.212204 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993ac9c-0aea-4adb-b952-7f2053aa5bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:51.213013 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993ac9c-0aea-4adb-b952-7f2053aa5bac admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:51.213602 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993ac9c-0aea-4adb-b952-7f2053aa5bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:51.213773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993ac9c-0aea-4adb-b952-7f2053aa5bac admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:51.214545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993ac9c-0aea-4adb-b952-7f2053aa5bac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:51.217047 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993ac9c-0aea-4adb-b952-7f2053aa5bac admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:51.233324 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1191/2350] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:51.234950 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0808606-13eb-406b-acde-d9e52e33711f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:51.243934 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c0808606-13eb-406b-acde-d9e52e33711f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:51.247420 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1160/2351] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:40:51 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:40:51.251525 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993ac9c-0aea-4adb-b952-7f2053aa5bac admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:51.253941 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2993ac9c-0aea-4adb-b952-7f2053aa5bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:51.262164 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:51.339486 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0808606-13eb-406b-acde-d9e52e33711f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:51.339698 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993ac9c-0aea-4adb-b952-7f2053aa5bac admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:51.342358 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df51e35-05ae-4c4e-a323-a75f30268ac7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:51.343063 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df51e35-05ae-4c4e-a323-a75f30268ac7 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:51.343323 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd2150-b1d3-4c35-87dd-3fbcfb102c95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:51.343446 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df51e35-05ae-4c4e-a323-a75f30268ac7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:51.344174 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df51e35-05ae-4c4e-a323-a75f30268ac7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:51.344863 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd2150-b1d3-4c35-87dd-3fbcfb102c95 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:51.345219 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd2150-b1d3-4c35-87dd-3fbcfb102c95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:51.345632 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd2150-b1d3-4c35-87dd-3fbcfb102c95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:51.355284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dd2150-b1d3-4c35-87dd-3fbcfb102c95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:51.356740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dd2150-b1d3-4c35-87dd-3fbcfb102c95 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:51.367876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1161/2352] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:40:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:40:51.382033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd2150-b1d3-4c35-87dd-3fbcfb102c95 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:51.404086 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd2150-b1d3-4c35-87dd-3fbcfb102c95 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:51.410225 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f54b04-eef1-4997-aa3e-078bf59be312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:51.411543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f54b04-eef1-4997-aa3e-078bf59be312 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:51.411836 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f54b04-eef1-4997-aa3e-078bf59be312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:51.412116 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f54b04-eef1-4997-aa3e-078bf59be312 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:51.412869 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f54b04-eef1-4997-aa3e-078bf59be312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:51.414531 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f54b04-eef1-4997-aa3e-078bf59be312 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:51.416111 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:51.425185 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df51e35-05ae-4c4e-a323-a75f30268ac7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:51.427738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df51e35-05ae-4c4e-a323-a75f30268ac7 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:51.444857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1162/2353] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:40:51.445011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1192/2354] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:40:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:40:51.445891 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f54b04-eef1-4997-aa3e-078bf59be312 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:51.465760 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f54b04-eef1-4997-aa3e-078bf59be312 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:51.471367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59beca9d-55ff-443c-91b1-5e76cb9440f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:51.472140 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59beca9d-55ff-443c-91b1-5e76cb9440f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:51.472715 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59beca9d-55ff-443c-91b1-5e76cb9440f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:51.473096 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59beca9d-55ff-443c-91b1-5e76cb9440f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:51.473896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59beca9d-55ff-443c-91b1-5e76cb9440f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:51.475820 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59beca9d-55ff-443c-91b1-5e76cb9440f4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:51.490697 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1163/2355] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:40:51.493802 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df51e35-05ae-4c4e-a323-a75f30268ac7 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:51.509701 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df51e35-05ae-4c4e-a323-a75f30268ac7 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:51.516322 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60cfbdd-a433-49a1-9641-fa0b4885041a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:51.517283 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cfbdd-a433-49a1-9641-fa0b4885041a admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:51.517568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cfbdd-a433-49a1-9641-fa0b4885041a admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:51.517838 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cfbdd-a433-49a1-9641-fa0b4885041a admin admin] PATH_INFO: `/v3/projects/21c11689a303465495b9209f70dfb664/users/95d4208cdd30408bae5bcdc4d9941cde/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:51.535543 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60cfbdd-a433-49a1-9641-fa0b4885041a admin admin] RBAC: Authorizing `identity:create_grant(project_id=21c11689a303465495b9209f70dfb664, user_id=95d4208cdd30408bae5bcdc4d9941cde, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:51.538318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60cfbdd-a433-49a1-9641-fa0b4885041a admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:51.554141 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1193/2356] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:40:51 2022] PUT /identity/v3/projects/21c11689a303465495b9209f70dfb664/users/95d4208cdd30408bae5bcdc4d9941cde/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:40:51.560148 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c9adc-0e84-4403-8135-fb3d45b13710 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:51.560148 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c9adc-0e84-4403-8135-fb3d45b13710 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:51.560455 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c9adc-0e84-4403-8135-fb3d45b13710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:51.585085 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8c3c9adc-0e84-4403-8135-fb3d45b13710 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:51.586555 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8c3c9adc-0e84-4403-8135-fb3d45b13710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:51.684188 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1164/2357] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:40:51.695635 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60cfbdd-a433-49a1-9641-fa0b4885041a admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:51.717387 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c9adc-0e84-4403-8135-fb3d45b13710 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:51.720027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8c3c9adc-0e84-4403-8135-fb3d45b13710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:51.763466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c9adc-0e84-4403-8135-fb3d45b13710 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:51.773699 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e18302-50af-4209-b6eb-ae747f50ee0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:51.775352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e18302-50af-4209-b6eb-ae747f50ee0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:51.777412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e18302-50af-4209-b6eb-ae747f50ee0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:51.777746 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e18302-50af-4209-b6eb-ae747f50ee0a admin admin] PATH_INFO: `/v3/projects/30ba4559e7f8482e9c28b9735efc770d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:51.786685 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e18302-50af-4209-b6eb-ae747f50ee0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=30ba4559e7f8482e9c28b9735efc770d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:51.786685 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e18302-50af-4209-b6eb-ae747f50ee0a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:51.808413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60cfbdd-a433-49a1-9641-fa0b4885041a admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:51.814714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de485fa1-044f-481c-8fa8-1f4413a1ccfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:51.815552 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de485fa1-044f-481c-8fa8-1f4413a1ccfe service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:51.815980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de485fa1-044f-481c-8fa8-1f4413a1ccfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:51.816601 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de485fa1-044f-481c-8fa8-1f4413a1ccfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:51.829172 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de485fa1-044f-481c-8fa8-1f4413a1ccfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:51.830843 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de485fa1-044f-481c-8fa8-1f4413a1ccfe service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:51.842285 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1194/2358] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:40:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:40:51.846776 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-27e18302-50af-4209-b6eb-ae747f50ee0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30ba4559e7f8482e9c28b9735efc770d', 'request_id': 'req-27e18302-50af-4209-b6eb-ae747f50ee0a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:51.861133 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-27e18302-50af-4209-b6eb-ae747f50ee0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30ba4559e7f8482e9c28b9735efc770d', 'request_id': 'req-27e18302-50af-4209-b6eb-ae747f50ee0a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:51.876920 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-27e18302-50af-4209-b6eb-ae747f50ee0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30ba4559e7f8482e9c28b9735efc770d', 'request_id': 'req-27e18302-50af-4209-b6eb-ae747f50ee0a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:51.904422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-27e18302-50af-4209-b6eb-ae747f50ee0a admin admin] The token cache is being invalidate because project 30ba4559e7f8482e9c28b9735efc770d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:40:51.904726 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-27e18302-50af-4209-b6eb-ae747f50ee0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:51.905446 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-27e18302-50af-4209-b6eb-ae747f50ee0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:40:51.922972 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1165/2359] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:40:51 2022] DELETE /identity/v3/projects/30ba4559e7f8482e9c28b9735efc770d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:40:56.346145 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a2b85-af56-42b9-b21e-56789299dc95 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:56.346997 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a2b85-af56-42b9-b21e-56789299dc95 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:56.347427 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a2b85-af56-42b9-b21e-56789299dc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:56.371940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-035a2b85-af56-42b9-b21e-56789299dc95 None None] MFA Rules not processed for user `0b7d100f2407432381d119c25814fff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:56.373599 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-035a2b85-af56-42b9-b21e-56789299dc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:56.432104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:56.476798 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1195/2360] 10.222.0.67 () {58 vars in 1009 bytes} [Tue May 31 14:40:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:56.529671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e18302-50af-4209-b6eb-ae747f50ee0a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:56.531764 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-27e18302-50af-4209-b6eb-ae747f50ee0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:56.626489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e18302-50af-4209-b6eb-ae747f50ee0a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:56.640838 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861e790-6cba-4751-89bc-f1e551f04031 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfa2282f957941b98db1d7497da3351b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:56.641543 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861e790-6cba-4751-89bc-f1e551f04031 service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:56.641841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861e790-6cba-4751-89bc-f1e551f04031 service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:56.642129 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861e790-6cba-4751-89bc-f1e551f04031 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:56.658236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861e790-6cba-4751-89bc-f1e551f04031 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:56.659991 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861e790-6cba-4751-89bc-f1e551f04031 service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:56.665101 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:56.680128 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1166/2361] 10.222.0.67 () {60 vars in 1384 bytes} [Tue May 31 14:40:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 31 14:40:57.643588 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cfbb09-b5a8-4b2a-9ad5-26bd50550f26 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:57.644491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cfbb09-b5a8-4b2a-9ad5-26bd50550f26 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:57.644491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cfbb09-b5a8-4b2a-9ad5-26bd50550f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:57.688175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-19cfbb09-b5a8-4b2a-9ad5-26bd50550f26 None None] MFA Rules not processed for user `0b7d100f2407432381d119c25814fff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:57.689446 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-19cfbb09-b5a8-4b2a-9ad5-26bd50550f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:57.716932 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1196/2362] 10.222.0.67 () {58 vars in 967 bytes} [Tue May 31 14:40:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:40:57.732761 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861e790-6cba-4751-89bc-f1e551f04031 service swift] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:57.747345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861e790-6cba-4751-89bc-f1e551f04031 service swift] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:57.763415 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6da333-2de4-40d5-b301-60496bc1e274 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfa2282f957941b98db1d7497da3351b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:57.764108 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6da333-2de4-40d5-b301-60496bc1e274 service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:57.764399 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6da333-2de4-40d5-b301-60496bc1e274 service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:57.764684 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6da333-2de4-40d5-b301-60496bc1e274 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:57.792983 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6da333-2de4-40d5-b301-60496bc1e274 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:57.800106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6da333-2de4-40d5-b301-60496bc1e274 service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:57.814781 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1167/2363] 10.222.0.67 () {60 vars in 1384 bytes} [Tue May 31 14:40:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 31 14:40:57.991696 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0fdfef-78ff-4a12-8c67-0489b272454c None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:57.991979 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0fdfef-78ff-4a12-8c67-0489b272454c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:57.992397 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0fdfef-78ff-4a12-8c67-0489b272454c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:58.059152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-fc0fdfef-78ff-4a12-8c67-0489b272454c None None] MFA Rules not processed for user `95d4208cdd30408bae5bcdc4d9941cde`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:58.060532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-fc0fdfef-78ff-4a12-8c67-0489b272454c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:58.231612 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1197/2364] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:40:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:58.236459 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6da333-2de4-40d5-b301-60496bc1e274 service swift] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:58.238016 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bd6da333-2de4-40d5-b301-60496bc1e274 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:58.261098 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba43d36-21a7-41d1-bd91-e3554e29351b None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:58.261098 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba43d36-21a7-41d1-bd91-e3554e29351b None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:58.261098 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba43d36-21a7-41d1-bd91-e3554e29351b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:58.297243 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-9ba43d36-21a7-41d1-bd91-e3554e29351b None None] MFA Rules not processed for user `0b7d100f2407432381d119c25814fff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:58.299012 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9ba43d36-21a7-41d1-bd91-e3554e29351b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:58.313676 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1198/2365] 10.222.0.67 () {58 vars in 1009 bytes} [Tue May 31 14:40:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:40:58.322225 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba43d36-21a7-41d1-bd91-e3554e29351b None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:58.338197 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba43d36-21a7-41d1-bd91-e3554e29351b None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:58.338375 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6da333-2de4-40d5-b301-60496bc1e274 service swift] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:58.346289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79243e-7ec1-4987-8f16-8674969b4281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:58.346988 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79243e-7ec1-4987-8f16-8674969b4281 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:58.347518 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79243e-7ec1-4987-8f16-8674969b4281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:58.348715 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79243e-7ec1-4987-8f16-8674969b4281 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:58.349753 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff79243e-7ec1-4987-8f16-8674969b4281 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:58.351709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff79243e-7ec1-4987-8f16-8674969b4281 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:58.354875 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e80a0-6a12-42d5-a249-f9985baed91e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfa2282f957941b98db1d7497da3351b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:58.355552 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e80a0-6a12-42d5-a249-f9985baed91e service swift] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:58.355842 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e80a0-6a12-42d5-a249-f9985baed91e service swift] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:58.356191 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e80a0-6a12-42d5-a249-f9985baed91e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:58.367980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e80a0-6a12-42d5-a249-f9985baed91e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:58.369654 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e80a0-6a12-42d5-a249-f9985baed91e service swift] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:58.383014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1199/2366] 10.222.0.67 () {60 vars in 1384 bytes} [Tue May 31 14:40:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 31 14:40:58.387085 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1168/2367] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:40:58 2022] POST /identity/v3/projects => generated 377 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:40:58.389303 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e80a0-6a12-42d5-a249-f9985baed91e service swift] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:58.403587 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e80a0-6a12-42d5-a249-f9985baed91e service swift] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:58.517741 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812811d8-4ee3-41c6-9d34-f88696de6625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:58.519120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812811d8-4ee3-41c6-9d34-f88696de6625 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:58.519617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812811d8-4ee3-41c6-9d34-f88696de6625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:58.520094 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812811d8-4ee3-41c6-9d34-f88696de6625 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:58.520985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812811d8-4ee3-41c6-9d34-f88696de6625 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:58.524603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812811d8-4ee3-41c6-9d34-f88696de6625 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:58.560410 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1200/2368] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:40:58 2022] POST /identity/v3/users => generated 295 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:40:58.561332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79243e-7ec1-4987-8f16-8674969b4281 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:58.577446 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79243e-7ec1-4987-8f16-8674969b4281 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:58.592530 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db55077-79d1-4270-9df0-fb9ffdf2f582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:58.593136 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db55077-79d1-4270-9df0-fb9ffdf2f582 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:58.593965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db55077-79d1-4270-9df0-fb9ffdf2f582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:58.594323 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db55077-79d1-4270-9df0-fb9ffdf2f582 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:58.595351 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db55077-79d1-4270-9df0-fb9ffdf2f582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:58.597076 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db55077-79d1-4270-9df0-fb9ffdf2f582 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:58.632569 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812811d8-4ee3-41c6-9d34-f88696de6625 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:58.644851 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812811d8-4ee3-41c6-9d34-f88696de6625 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:58.645809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1169/2369] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:40:58.660144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d7f3b-4ec7-483d-9c6f-3a099b501a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:58.661078 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d7f3b-4ec7-483d-9c6f-3a099b501a33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:58.668242 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d7f3b-4ec7-483d-9c6f-3a099b501a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:58.668858 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d7f3b-4ec7-483d-9c6f-3a099b501a33 admin admin] PATH_INFO: `/v3/projects/797ac2d6a8344b7ba939e3b078f44b0b/users/11f1f384f2a241de835900a7fef57731/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:58.712439 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d7f3b-4ec7-483d-9c6f-3a099b501a33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=797ac2d6a8344b7ba939e3b078f44b0b, user_id=11f1f384f2a241de835900a7fef57731, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:58.715362 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d7f3b-4ec7-483d-9c6f-3a099b501a33 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:58.740509 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1201/2370] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:40:58 2022] PUT /identity/v3/projects/797ac2d6a8344b7ba939e3b078f44b0b/users/11f1f384f2a241de835900a7fef57731/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:40:58.744468 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db55077-79d1-4270-9df0-fb9ffdf2f582 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:58.771342 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db55077-79d1-4270-9df0-fb9ffdf2f582 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:58.959487 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7c20d-2754-4f33-9a53-d6e4ecb58079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:58.960472 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7c20d-2754-4f33-9a53-d6e4ecb58079 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:58.960914 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7c20d-2754-4f33-9a53-d6e4ecb58079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:58.961346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7c20d-2754-4f33-9a53-d6e4ecb58079 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:58.962246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7c20d-2754-4f33-9a53-d6e4ecb58079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:58.964032 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7c20d-2754-4f33-9a53-d6e4ecb58079 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:58.993491 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1170/2371] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:40:58.995968 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d7f3b-4ec7-483d-9c6f-3a099b501a33 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:59.016712 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d7f3b-4ec7-483d-9c6f-3a099b501a33 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:59.045635 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd890b58-49c2-4d3b-994c-46628ac058d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:59.047778 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd890b58-49c2-4d3b-994c-46628ac058d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:59.048240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd890b58-49c2-4d3b-994c-46628ac058d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:59.048688 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd890b58-49c2-4d3b-994c-46628ac058d2 admin admin] PATH_INFO: `/v3/projects/797ac2d6a8344b7ba939e3b078f44b0b/users/11f1f384f2a241de835900a7fef57731/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:59.053223 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd890b58-49c2-4d3b-994c-46628ac058d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=797ac2d6a8344b7ba939e3b078f44b0b, user_id=11f1f384f2a241de835900a7fef57731, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:59.060239 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd890b58-49c2-4d3b-994c-46628ac058d2 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:59.087557 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:59.101059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1202/2372] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:40:58 2022] PUT /identity/v3/projects/797ac2d6a8344b7ba939e3b078f44b0b/users/11f1f384f2a241de835900a7fef57731/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:40:59.173739 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa229a-30d7-4c73-b881-b06593fb4c32 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:59.174300 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa229a-30d7-4c73-b881-b06593fb4c32 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:59.174768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa229a-30d7-4c73-b881-b06593fb4c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:59.233274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8caa229a-30d7-4c73-b881-b06593fb4c32 None None] MFA Rules not processed for user `0b7d100f2407432381d119c25814fff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:40:59.236359 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8caa229a-30d7-4c73-b881-b06593fb4c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:40:59.338371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:40:59.373838 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1171/2373] 10.222.0.67 () {58 vars in 967 bytes} [Tue May 31 14:40:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:40:59.388365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd890b58-49c2-4d3b-994c-46628ac058d2 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:40:59.569348 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd890b58-49c2-4d3b-994c-46628ac058d2 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:40:59.581582 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a9be4-cbf5-4632-9883-ecad37e756ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfa2282f957941b98db1d7497da3351b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:40:59.583831 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a9be4-cbf5-4632-9883-ecad37e756ca service swift] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:40:59.584124 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a9be4-cbf5-4632-9883-ecad37e756ca service swift] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:40:59.584908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a9be4-cbf5-4632-9883-ecad37e756ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:40:59.622281 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a9be4-cbf5-4632-9883-ecad37e756ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:40:59.627634 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a9be4-cbf5-4632-9883-ecad37e756ca service swift] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:40:59.647873 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1203/2374] 10.222.0.67 () {60 vars in 1384 bytes} [Tue May 31 14:40:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 31 14:41:02.707552 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa229a-30d7-4c73-b881-b06593fb4c32 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:41:02.709811 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8caa229a-30d7-4c73-b881-b06593fb4c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:41:02.802230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa229a-30d7-4c73-b881-b06593fb4c32 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:41:02.817217 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0faea-19e3-4247-a839-bd249ff4508e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:41:02.818839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0faea-19e3-4247-a839-bd249ff4508e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:02.819130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0faea-19e3-4247-a839-bd249ff4508e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:02.819411 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0faea-19e3-4247-a839-bd249ff4508e admin admin] PATH_INFO: `/v3/users/ab5d06b0a3324dab903f485342590db9` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:02.820775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c0faea-19e3-4247-a839-bd249ff4508e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab5d06b0a3324dab903f485342590db9)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:41:02.822919 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c0faea-19e3-4247-a839-bd249ff4508e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:41:02.917824 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c3c0faea-19e3-4247-a839-bd249ff4508e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab5d06b0a3324dab903f485342590db9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:02.939087 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c3c0faea-19e3-4247-a839-bd249ff4508e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab5d06b0a3324dab903f485342590db9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:02.950150 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c3c0faea-19e3-4247-a839-bd249ff4508e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab5d06b0a3324dab903f485342590db9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:02.986255 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c3c0faea-19e3-4247-a839-bd249ff4508e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab5d06b0a3324dab903f485342590db9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:02.989785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c3c0faea-19e3-4247-a839-bd249ff4508e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab5d06b0a3324dab903f485342590db9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:03.013033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1172/2375] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:41:02 2022] DELETE /identity/v3/users/ab5d06b0a3324dab903f485342590db9 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:41:03.016193 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4e893-36cc-41ce-a4b1-e9b4d19fca7a None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:03.016193 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4e893-36cc-41ce-a4b1-e9b4d19fca7a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:03.016748 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4e893-36cc-41ce-a4b1-e9b4d19fca7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:03.059065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-85b4e893-36cc-41ce-a4b1-e9b4d19fca7a None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:41:03.060333 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-85b4e893-36cc-41ce-a4b1-e9b4d19fca7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:41:03.232852 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1204/2376] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:41:03.244379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0faea-19e3-4247-a839-bd249ff4508e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:41:03.245908 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c3c0faea-19e3-4247-a839-bd249ff4508e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:41:03.358261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0faea-19e3-4247-a839-bd249ff4508e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:41:03.365483 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9aa893-5ff4-4faf-866c-818e453827a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:41:03.366737 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9aa893-5ff4-4faf-866c-818e453827a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:03.367018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9aa893-5ff4-4faf-866c-818e453827a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:03.367412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9aa893-5ff4-4faf-866c-818e453827a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:03.386126 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9aa893-5ff4-4faf-866c-818e453827a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:41:03.388448 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9aa893-5ff4-4faf-866c-818e453827a9 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:41:03.408134 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1173/2377] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:41:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:41:03.734382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4e893-36cc-41ce-a4b1-e9b4d19fca7a None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:41:03.735870 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-85b4e893-36cc-41ce-a4b1-e9b4d19fca7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:41:03.760637 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4e893-36cc-41ce-a4b1-e9b4d19fca7a None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:41:03.780457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0153b-a3d4-4b3e-b141-4b469a688a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:41:03.781812 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0153b-a3d4-4b3e-b141-4b469a688a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:03.782273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0153b-a3d4-4b3e-b141-4b469a688a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:03.782774 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0153b-a3d4-4b3e-b141-4b469a688a8a admin admin] PATH_INFO: `/v3/projects/7dadc6972be242a48c64084726085851` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:03.784262 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb0153b-a3d4-4b3e-b141-4b469a688a8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dadc6972be242a48c64084726085851)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:41:03.787669 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb0153b-a3d4-4b3e-b141-4b469a688a8a admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:41:03.900632 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-bbb0153b-a3d4-4b3e-b141-4b469a688a8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dadc6972be242a48c64084726085851', 'request_id': 'req-bbb0153b-a3d4-4b3e-b141-4b469a688a8a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:03.914602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-bbb0153b-a3d4-4b3e-b141-4b469a688a8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dadc6972be242a48c64084726085851', 'request_id': 'req-bbb0153b-a3d4-4b3e-b141-4b469a688a8a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:03.929393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-bbb0153b-a3d4-4b3e-b141-4b469a688a8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dadc6972be242a48c64084726085851', 'request_id': 'req-bbb0153b-a3d4-4b3e-b141-4b469a688a8a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:03.982188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-bbb0153b-a3d4-4b3e-b141-4b469a688a8a admin admin] The token cache is being invalidate because project 7dadc6972be242a48c64084726085851 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:41:03.983101 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-bbb0153b-a3d4-4b3e-b141-4b469a688a8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:03.984947 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-bbb0153b-a3d4-4b3e-b141-4b469a688a8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:03.993008 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1205/2378] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:41:03 2022] DELETE /identity/v3/projects/7dadc6972be242a48c64084726085851 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:41:03.995867 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7503889-94be-49e5-b622-788bf15dc2ed None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:03.996999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7503889-94be-49e5-b622-788bf15dc2ed None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:03.997296 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7503889-94be-49e5-b622-788bf15dc2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:04.039196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:41:04.068472 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c7503889-94be-49e5-b622-788bf15dc2ed None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:41:04.070217 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c7503889-94be-49e5-b622-788bf15dc2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:41:04.185633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1174/2379] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:41:04.193307 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e086393-f06e-4d42-a80e-5b6129842f0c None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:04.194365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e086393-f06e-4d42-a80e-5b6129842f0c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:04.194860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e086393-f06e-4d42-a80e-5b6129842f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:04.221000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-1e086393-f06e-4d42-a80e-5b6129842f0c None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:41:04.223679 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1e086393-f06e-4d42-a80e-5b6129842f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:41:04.234145 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1206/2380] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:41:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:41:04.237962 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7503889-94be-49e5-b622-788bf15dc2ed None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:41:04.250535 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7503889-94be-49e5-b622-788bf15dc2ed None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:41:04.263623 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ad3ff-d770-477e-8d8d-7587268745e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:41:04.274385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ad3ff-d770-477e-8d8d-7587268745e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:04.275466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ad3ff-d770-477e-8d8d-7587268745e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:04.276603 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ad3ff-d770-477e-8d8d-7587268745e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:04.277018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ad3ff-d770-477e-8d8d-7587268745e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:41:04.277552 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ad3ff-d770-477e-8d8d-7587268745e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:41:04.291778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ad3ff-d770-477e-8d8d-7587268745e1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:41:04.317192 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e086393-f06e-4d42-a80e-5b6129842f0c None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:41:04.319880 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1e086393-f06e-4d42-a80e-5b6129842f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:41:04.335706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1175/2381] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 31 14:41:04.340964 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ad3ff-d770-477e-8d8d-7587268745e1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:41:04.352744 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e086393-f06e-4d42-a80e-5b6129842f0c None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:41:04.365188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d925b54-f073-4816-bd30-e89fc9971b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:41:04.366590 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d925b54-f073-4816-bd30-e89fc9971b42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:04.366902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d925b54-f073-4816-bd30-e89fc9971b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:04.367186 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d925b54-f073-4816-bd30-e89fc9971b42 admin admin] PATH_INFO: `/v3/users/0ccc5f10cf584b8ca5bdc158a02c2ce1` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:04.368538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ad3ff-d770-477e-8d8d-7587268745e1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:41:04.371479 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d925b54-f073-4816-bd30-e89fc9971b42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ccc5f10cf584b8ca5bdc158a02c2ce1)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:41:04.377745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d925b54-f073-4816-bd30-e89fc9971b42 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:41:04.378458 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51703b1-73e4-40db-bb4d-f43a0e5a2fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:41:04.380394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51703b1-73e4-40db-bb4d-f43a0e5a2fd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:04.380680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51703b1-73e4-40db-bb4d-f43a0e5a2fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:04.380955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51703b1-73e4-40db-bb4d-f43a0e5a2fd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:04.381943 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:41:04.382209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51703b1-73e4-40db-bb4d-f43a0e5a2fd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:41:04.386764 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51703b1-73e4-40db-bb4d-f43a0e5a2fd4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:41:04.469753 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1176/2382] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:41:04 2022] POST /identity/v3/projects => generated 391 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:41:04.471513 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51703b1-73e4-40db-bb4d-f43a0e5a2fd4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:41:04.509347 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4d925b54-f073-4816-bd30-e89fc9971b42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ccc5f10cf584b8ca5bdc158a02c2ce1'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:04.533332 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4d925b54-f073-4816-bd30-e89fc9971b42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ccc5f10cf584b8ca5bdc158a02c2ce1'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:04.534594 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4d925b54-f073-4816-bd30-e89fc9971b42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ccc5f10cf584b8ca5bdc158a02c2ce1'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:04.564821 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4d925b54-f073-4816-bd30-e89fc9971b42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ccc5f10cf584b8ca5bdc158a02c2ce1'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:04.566000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4d925b54-f073-4816-bd30-e89fc9971b42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ccc5f10cf584b8ca5bdc158a02c2ce1'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:04.589403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1207/2383] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:41:04 2022] DELETE /identity/v3/users/0ccc5f10cf584b8ca5bdc158a02c2ce1 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:41:04.604773 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af6595-961b-4da0-a2b5-2f3a5252e0c2 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:04.605079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af6595-961b-4da0-a2b5-2f3a5252e0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:04.605367 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af6595-961b-4da0-a2b5-2f3a5252e0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:04.643687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51703b1-73e4-40db-bb4d-f43a0e5a2fd4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:41:04.663594 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-34af6595-961b-4da0-a2b5-2f3a5252e0c2 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:41:04.664888 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-34af6595-961b-4da0-a2b5-2f3a5252e0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:41:04.786824 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1208/2384] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:41:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:41:04.790082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642b992-5216-4c8e-a8d6-5fabc88f33dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:41:04.791103 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642b992-5216-4c8e-a8d6-5fabc88f33dc admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:04.791664 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642b992-5216-4c8e-a8d6-5fabc88f33dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:04.792676 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642b992-5216-4c8e-a8d6-5fabc88f33dc admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:04.793952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642b992-5216-4c8e-a8d6-5fabc88f33dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:41:04.796281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642b992-5216-4c8e-a8d6-5fabc88f33dc admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:41:04.799428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af6595-961b-4da0-a2b5-2f3a5252e0c2 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:41:04.801215 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-34af6595-961b-4da0-a2b5-2f3a5252e0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:41:04.830426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1177/2385] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:41:04 2022] POST /identity/v3/users => generated 296 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:41:04.831056 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642b992-5216-4c8e-a8d6-5fabc88f33dc admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:41:04.833005 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3642b992-5216-4c8e-a8d6-5fabc88f33dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:41:04.848008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642b992-5216-4c8e-a8d6-5fabc88f33dc admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:41:04.855766 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1caaaf-b0d9-41af-8b80-dcdac984d741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:41:04.856752 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1caaaf-b0d9-41af-8b80-dcdac984d741 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:04.857043 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1caaaf-b0d9-41af-8b80-dcdac984d741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:04.857326 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1caaaf-b0d9-41af-8b80-dcdac984d741 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:04.858418 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1caaaf-b0d9-41af-8b80-dcdac984d741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:41:04.860809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1caaaf-b0d9-41af-8b80-dcdac984d741 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:41:04.862393 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:41:04.880015 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1178/2386] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:41:04.881397 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1caaaf-b0d9-41af-8b80-dcdac984d741 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:41:04.896612 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1caaaf-b0d9-41af-8b80-dcdac984d741 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:41:04.902578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a92da8-73cf-4d3b-b470-32b1019d4b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:41:04.903308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a92da8-73cf-4d3b-b470-32b1019d4b3e admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:04.903596 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a92da8-73cf-4d3b-b470-32b1019d4b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:04.904247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a92da8-73cf-4d3b-b470-32b1019d4b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:04.905262 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a92da8-73cf-4d3b-b470-32b1019d4b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:41:04.907648 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a92da8-73cf-4d3b-b470-32b1019d4b3e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:41:04.921465 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1179/2387] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:41:04.925782 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a92da8-73cf-4d3b-b470-32b1019d4b3e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:41:04.928605 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af6595-961b-4da0-a2b5-2f3a5252e0c2 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:41:04.938765 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae009a-39e2-4aa3-a8da-7733ad006758 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:41:04.940378 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae009a-39e2-4aa3-a8da-7733ad006758 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:04.941281 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae009a-39e2-4aa3-a8da-7733ad006758 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:04.941758 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae009a-39e2-4aa3-a8da-7733ad006758 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:04.943118 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a92da8-73cf-4d3b-b470-32b1019d4b3e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:41:04.953137 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b92e2-64e3-46bf-89a5-90e1b454a898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:41:04.954897 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b92e2-64e3-46bf-89a5-90e1b454a898 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:04.955624 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b92e2-64e3-46bf-89a5-90e1b454a898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:04.956349 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ae009a-39e2-4aa3-a8da-7733ad006758 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:41:04.956727 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b92e2-64e3-46bf-89a5-90e1b454a898 admin admin] PATH_INFO: `/v3/projects/e077b16fa5fb46fb84ba46929b6267b6/users/58934ab05a4f4a629a1ff63bd7cd9f8f/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:04.958265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ae009a-39e2-4aa3-a8da-7733ad006758 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:41:04.977314 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1209/2388] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:41:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:41:04.989604 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b92e2-64e3-46bf-89a5-90e1b454a898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e077b16fa5fb46fb84ba46929b6267b6, user_id=58934ab05a4f4a629a1ff63bd7cd9f8f, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:41:04.993453 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b92e2-64e3-46bf-89a5-90e1b454a898 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:41:05.021457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1180/2389] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:41:04 2022] PUT /identity/v3/projects/e077b16fa5fb46fb84ba46929b6267b6/users/58934ab05a4f4a629a1ff63bd7cd9f8f/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:41:05.034452 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5fde6-9680-40ca-9f91-57b8175d678a None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:05.034846 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5fde6-9680-40ca-9f91-57b8175d678a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:05.035139 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5fde6-9680-40ca-9f91-57b8175d678a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:05.078746 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-9dd5fde6-9680-40ca-9f91-57b8175d678a None None] MFA Rules not processed for user `58934ab05a4f4a629a1ff63bd7cd9f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:41:05.080009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9dd5fde6-9680-40ca-9f91-57b8175d678a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:41:05.100852 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:41:05.206874 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1210/2390] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:41:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:41:05.216289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b92e2-64e3-46bf-89a5-90e1b454a898 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:41:05.219325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-795b92e2-64e3-46bf-89a5-90e1b454a898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:41:05.301534 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b92e2-64e3-46bf-89a5-90e1b454a898 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:41:05.305689 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca2da5-e039-4c5b-957c-ea8ed6128ba2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:41:05.306469 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca2da5-e039-4c5b-957c-ea8ed6128ba2 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:05.306717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca2da5-e039-4c5b-957c-ea8ed6128ba2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:05.306953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca2da5-e039-4c5b-957c-ea8ed6128ba2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:05.318459 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca2da5-e039-4c5b-957c-ea8ed6128ba2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:41:05.319955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca2da5-e039-4c5b-957c-ea8ed6128ba2 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:41:05.342678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1181/2391] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:41:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:41:05.556902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5fde6-9680-40ca-9f91-57b8175d678a None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:41:05.558389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9dd5fde6-9680-40ca-9f91-57b8175d678a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:41:05.710373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5fde6-9680-40ca-9f91-57b8175d678a None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:41:05.717544 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f9ad7-ecc4-4681-82be-8d1e1a456149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:41:05.719444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f9ad7-ecc4-4681-82be-8d1e1a456149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:05.719919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f9ad7-ecc4-4681-82be-8d1e1a456149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:05.720457 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f9ad7-ecc4-4681-82be-8d1e1a456149 admin admin] PATH_INFO: `/v3/projects/7ea9415341504da0a28fa1ac2deba1ca` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:05.723232 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f9ad7-ecc4-4681-82be-8d1e1a456149 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ea9415341504da0a28fa1ac2deba1ca)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:41:05.730504 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f9ad7-ecc4-4681-82be-8d1e1a456149 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:41:05.820918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-933f9ad7-ecc4-4681-82be-8d1e1a456149 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ea9415341504da0a28fa1ac2deba1ca', 'request_id': 'req-933f9ad7-ecc4-4681-82be-8d1e1a456149'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:05.829519 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-933f9ad7-ecc4-4681-82be-8d1e1a456149 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ea9415341504da0a28fa1ac2deba1ca', 'request_id': 'req-933f9ad7-ecc4-4681-82be-8d1e1a456149'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:05.838910 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-933f9ad7-ecc4-4681-82be-8d1e1a456149 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ea9415341504da0a28fa1ac2deba1ca', 'request_id': 'req-933f9ad7-ecc4-4681-82be-8d1e1a456149'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:05.873957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-933f9ad7-ecc4-4681-82be-8d1e1a456149 admin admin] The token cache is being invalidate because project 7ea9415341504da0a28fa1ac2deba1ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:41:05.875053 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-933f9ad7-ecc4-4681-82be-8d1e1a456149 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:05.875912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-933f9ad7-ecc4-4681-82be-8d1e1a456149 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:05.878934 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1211/2392] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:41:05 2022] DELETE /identity/v3/projects/7ea9415341504da0a28fa1ac2deba1ca => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:41:06.405231 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c102a2-8515-41c9-9279-a8cd921ce99c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:06.406581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c102a2-8515-41c9-9279-a8cd921ce99c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:06.406581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c102a2-8515-41c9-9279-a8cd921ce99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:06.448214 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b1c102a2-8515-41c9-9279-a8cd921ce99c None None] MFA Rules not processed for user `11f1f384f2a241de835900a7fef57731`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:41:06.449790 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b1c102a2-8515-41c9-9279-a8cd921ce99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:41:06.551548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1182/2393] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:41:06 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:41:06.566355 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f9ad7-ecc4-4681-82be-8d1e1a456149 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:41:06.567897 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-933f9ad7-ecc4-4681-82be-8d1e1a456149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:41:06.620922 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:41:06.657335 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f9ad7-ecc4-4681-82be-8d1e1a456149 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:41:06.663264 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72588b-3dbd-4dd2-9716-1c4e12b1e521 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:41:06.663938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72588b-3dbd-4dd2-9716-1c4e12b1e521 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:06.664226 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72588b-3dbd-4dd2-9716-1c4e12b1e521 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:06.664507 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72588b-3dbd-4dd2-9716-1c4e12b1e521 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:06.666390 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cf72588b-3dbd-4dd2-9716-1c4e12b1e521 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:41:06.735151 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf72588b-3dbd-4dd2-9716-1c4e12b1e521 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:41:06.736881 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf72588b-3dbd-4dd2-9716-1c4e12b1e521 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:41:06.747736 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1212/2394] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:41:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:41:16.547326 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=77,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:41:16.742206 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=37 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:41:22.752672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0561f-bc8b-474b-8b8a-8539acf323fc None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:22.753257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0561f-bc8b-474b-8b8a-8539acf323fc None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:22.753257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0561f-bc8b-474b-8b8a-8539acf323fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:22.773054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6ce0561f-bc8b-474b-8b8a-8539acf323fc None None] MFA Rules not processed for user `91255c0b2d914ef9be10eb69d7d8ed4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:41:22.774292 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6ce0561f-bc8b-474b-8b8a-8539acf323fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:41:22.835224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1183/2395] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:41:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:41:32.828309 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=25 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:41:34.616183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72588b-3dbd-4dd2-9716-1c4e12b1e521 service nova] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:41:34.617676 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cf72588b-3dbd-4dd2-9716-1c4e12b1e521 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:41:34.686659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72588b-3dbd-4dd2-9716-1c4e12b1e521 service nova] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:41:34.691452 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd29dd3-cf97-4032-ae96-466fc4daeba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:41:34.692317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd29dd3-cf97-4032-ae96-466fc4daeba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:34.692597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd29dd3-cf97-4032-ae96-466fc4daeba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:34.692869 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd29dd3-cf97-4032-ae96-466fc4daeba0 admin admin] PATH_INFO: `/v3/users/58934ab05a4f4a629a1ff63bd7cd9f8f` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:34.694040 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd29dd3-cf97-4032-ae96-466fc4daeba0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58934ab05a4f4a629a1ff63bd7cd9f8f)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:41:34.695981 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd29dd3-cf97-4032-ae96-466fc4daeba0 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:41:34.743541 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-edd29dd3-cf97-4032-ae96-466fc4daeba0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58934ab05a4f4a629a1ff63bd7cd9f8f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:34.750778 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-edd29dd3-cf97-4032-ae96-466fc4daeba0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58934ab05a4f4a629a1ff63bd7cd9f8f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:34.751429 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-edd29dd3-cf97-4032-ae96-466fc4daeba0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58934ab05a4f4a629a1ff63bd7cd9f8f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:34.763098 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-edd29dd3-cf97-4032-ae96-466fc4daeba0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58934ab05a4f4a629a1ff63bd7cd9f8f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:34.763729 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-edd29dd3-cf97-4032-ae96-466fc4daeba0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58934ab05a4f4a629a1ff63bd7cd9f8f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:34.775595 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1213/2396] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:41:34 2022] DELETE /identity/v3/users/58934ab05a4f4a629a1ff63bd7cd9f8f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:41:34.780583 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948369e-fa7b-4f60-8776-22117d7940fb None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:34.781097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948369e-fa7b-4f60-8776-22117d7940fb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:34.781207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948369e-fa7b-4f60-8776-22117d7940fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:34.799671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7948369e-fa7b-4f60-8776-22117d7940fb None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:41:34.800868 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7948369e-fa7b-4f60-8776-22117d7940fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:41:34.859919 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1184/2397] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:41:34.869398 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd29dd3-cf97-4032-ae96-466fc4daeba0 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:41:34.870829 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-edd29dd3-cf97-4032-ae96-466fc4daeba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:41:34.931777 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd29dd3-cf97-4032-ae96-466fc4daeba0 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:41:34.935468 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a55f4-6a20-46d1-aa9b-45530ed18b1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:41:34.936121 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a55f4-6a20-46d1-aa9b-45530ed18b1f service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:34.936409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a55f4-6a20-46d1-aa9b-45530ed18b1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:34.936704 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a55f4-6a20-46d1-aa9b-45530ed18b1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:34.946679 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a55f4-6a20-46d1-aa9b-45530ed18b1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:41:34.948179 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a55f4-6a20-46d1-aa9b-45530ed18b1f service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:41:34.958791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1214/2398] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:41:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:41:35.243052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948369e-fa7b-4f60-8776-22117d7940fb None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:41:35.244588 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7948369e-fa7b-4f60-8776-22117d7940fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:41:35.264344 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948369e-fa7b-4f60-8776-22117d7940fb None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:41:35.270236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8156f4f-dc76-413f-9f4c-ef3a80e29ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:41:35.271250 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8156f4f-dc76-413f-9f4c-ef3a80e29ffa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:35.271604 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8156f4f-dc76-413f-9f4c-ef3a80e29ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:35.271952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8156f4f-dc76-413f-9f4c-ef3a80e29ffa admin admin] PATH_INFO: `/v3/projects/e077b16fa5fb46fb84ba46929b6267b6` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:35.273332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8156f4f-dc76-413f-9f4c-ef3a80e29ffa admin admin] RBAC: Authorizing `identity:delete_project(project_id=e077b16fa5fb46fb84ba46929b6267b6)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:41:35.275476 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8156f4f-dc76-413f-9f4c-ef3a80e29ffa admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:41:35.326049 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d8156f4f-dc76-413f-9f4c-ef3a80e29ffa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e077b16fa5fb46fb84ba46929b6267b6', 'request_id': 'req-d8156f4f-dc76-413f-9f4c-ef3a80e29ffa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:35.332834 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d8156f4f-dc76-413f-9f4c-ef3a80e29ffa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e077b16fa5fb46fb84ba46929b6267b6', 'request_id': 'req-d8156f4f-dc76-413f-9f4c-ef3a80e29ffa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:35.339554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d8156f4f-dc76-413f-9f4c-ef3a80e29ffa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e077b16fa5fb46fb84ba46929b6267b6', 'request_id': 'req-d8156f4f-dc76-413f-9f4c-ef3a80e29ffa'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:35.354213 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d8156f4f-dc76-413f-9f4c-ef3a80e29ffa admin admin] The token cache is being invalidate because project e077b16fa5fb46fb84ba46929b6267b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:41:35.354506 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d8156f4f-dc76-413f-9f4c-ef3a80e29ffa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:35.355235 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d8156f4f-dc76-413f-9f4c-ef3a80e29ffa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:41:35.356892 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1185/2399] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:41:35 2022] DELETE /identity/v3/projects/e077b16fa5fb46fb84ba46929b6267b6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:41:40.012923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a55f4-6a20-46d1-aa9b-45530ed18b1f service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:41:40.015085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cd3a55f4-6a20-46d1-aa9b-45530ed18b1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:41:40.042030 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:41:40.116920 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a55f4-6a20-46d1-aa9b-45530ed18b1f service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:41:40.179840 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b504636-60f6-407c-9b85-adf335e73a71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c1c13515a8b41a781478dc65e6a88b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:41:40.180495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b504636-60f6-407c-9b85-adf335e73a71 service placement] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:40.180781 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b504636-60f6-407c-9b85-adf335e73a71 service placement] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:40.181059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b504636-60f6-407c-9b85-adf335e73a71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:40.182916 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8b504636-60f6-407c-9b85-adf335e73a71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:41:40.195197 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b504636-60f6-407c-9b85-adf335e73a71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:41:40.196795 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b504636-60f6-407c-9b85-adf335e73a71 service placement] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:41:40.207081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1215/2400] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:41:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:41:42.336068 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca391034-adcd-4f1f-ba82-8220604813da None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:41:42.336643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca391034-adcd-4f1f-ba82-8220604813da None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:41:42.336643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca391034-adcd-4f1f-ba82-8220604813da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:41:42.359489 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ca391034-adcd-4f1f-ba82-8220604813da None None] MFA Rules not processed for user `95d4208cdd30408bae5bcdc4d9941cde`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:41:42.360634 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ca391034-adcd-4f1f-ba82-8220604813da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:41:42.423232 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1186/2401] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:41:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:41:50.202495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=57 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:41:52.415535 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:42:01.049984 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b504636-60f6-407c-9b85-adf335e73a71 service placement] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:42:01.051913 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8b504636-60f6-407c-9b85-adf335e73a71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:42:01.107503 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b504636-60f6-407c-9b85-adf335e73a71 service placement] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:42:01.111093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31087a36-454f-4e48-965d-6ab5dd6fc343 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:42:01.111730 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31087a36-454f-4e48-965d-6ab5dd6fc343 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:42:01.112015 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31087a36-454f-4e48-965d-6ab5dd6fc343 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:42:01.112291 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31087a36-454f-4e48-965d-6ab5dd6fc343 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:42:01.114548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-31087a36-454f-4e48-965d-6ab5dd6fc343 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:42:01.175711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31087a36-454f-4e48-965d-6ab5dd6fc343 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:42:01.177632 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31087a36-454f-4e48-965d-6ab5dd6fc343 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:42:01.187206 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1216/2402] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:42:01 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:42:11.182777 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=50 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:43:03.320930 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98024b-fba9-4b24-a207-21abe44dfc6b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:43:03.321452 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98024b-fba9-4b24-a207-21abe44dfc6b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:43:03.321544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98024b-fba9-4b24-a207-21abe44dfc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:43:03.370353 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-af98024b-fba9-4b24-a207-21abe44dfc6b None None] MFA Rules not processed for user `0b7d100f2407432381d119c25814fff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:43:03.371625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-af98024b-fba9-4b24-a207-21abe44dfc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:43:03.451499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1187/2403] 10.222.0.67 () {58 vars in 1009 bytes} [Tue May 31 14:43:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:43:03.501143 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31087a36-454f-4e48-965d-6ab5dd6fc343 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:43:03.502604 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-31087a36-454f-4e48-965d-6ab5dd6fc343 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:43:03.591438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31087a36-454f-4e48-965d-6ab5dd6fc343 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:43:03.596357 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc37f39-b275-408d-9871-84e883601621 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfa2282f957941b98db1d7497da3351b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:43:03.596993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc37f39-b275-408d-9871-84e883601621 service swift] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:43:03.597273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc37f39-b275-408d-9871-84e883601621 service swift] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:43:03.597545 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc37f39-b275-408d-9871-84e883601621 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:43:03.605951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc37f39-b275-408d-9871-84e883601621 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:43:03.607529 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc37f39-b275-408d-9871-84e883601621 service swift] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:43:03.621055 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1217/2404] 10.222.0.67 () {60 vars in 1384 bytes} [Tue May 31 14:43:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 31 14:43:13.448564 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=43 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:43:13.612145 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=41 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:43:17.094857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfaa104-85dc-434c-afb3-18b4e02d699b None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:43:17.095412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfaa104-85dc-434c-afb3-18b4e02d699b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:43:17.096222 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfaa104-85dc-434c-afb3-18b4e02d699b None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:43:17.097877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1188/2405] 10.222.0.67 () {54 vars in 921 bytes} [Tue May 31 14:43:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:43:17.105005 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb47336-db28-4e6a-8304-0d7883921274 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:43:17.105469 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb47336-db28-4e6a-8304-0d7883921274 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:43:17.105578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb47336-db28-4e6a-8304-0d7883921274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:43:17.157997 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-1fb47336-db28-4e6a-8304-0d7883921274 None None] MFA Rules not processed for user `68c6363b60954c3ebf50d2019f5f2fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:43:17.159163 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1fb47336-db28-4e6a-8304-0d7883921274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:43:17.238041 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1218/2406] 10.222.0.67 () {58 vars in 1018 bytes} [Tue May 31 14:43:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:43:17.250080 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfaa104-85dc-434c-afb3-18b4e02d699b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:43:17.251778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3cfaa104-85dc-434c-afb3-18b4e02d699b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:43:17.330117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfaa104-85dc-434c-afb3-18b4e02d699b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:43:17.333703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9109b2b5-5558-46c4-a946-7b7fd6f74a77 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8a4b0dd4e21414d91fe7bed478d15e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:43:17.334348 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109b2b5-5558-46c4-a946-7b7fd6f74a77 service glance] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:43:17.334631 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109b2b5-5558-46c4-a946-7b7fd6f74a77 service glance] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:43:17.334943 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109b2b5-5558-46c4-a946-7b7fd6f74a77 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:43:17.346071 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9109b2b5-5558-46c4-a946-7b7fd6f74a77 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:43:17.347532 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9109b2b5-5558-46c4-a946-7b7fd6f74a77 service glance] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:43:17.359286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1189/2407] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:43:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:43:17.428773 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b132c8b-6ec1-41c2-a614-b9a8251ab2ac None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:43:17.428989 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b132c8b-6ec1-41c2-a614-b9a8251ab2ac None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:43:17.429268 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b132c8b-6ec1-41c2-a614-b9a8251ab2ac None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:43:17.430688 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1219/2408] 10.222.0.67 () {54 vars in 921 bytes} [Tue May 31 14:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:43:17.435549 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdcbc98-3236-4237-9959-e3a55789d4ff None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:43:17.435872 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdcbc98-3236-4237-9959-e3a55789d4ff None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:43:17.436153 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdcbc98-3236-4237-9959-e3a55789d4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:43:17.453660 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2fdcbc98-3236-4237-9959-e3a55789d4ff None None] MFA Rules not processed for user `8966bc2ea7ff497c84214c9c9e6d53a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:43:17.454910 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2fdcbc98-3236-4237-9959-e3a55789d4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:43:17.461909 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1190/2409] 10.222.0.67 () {58 vars in 1018 bytes} [Tue May 31 14:43:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:43:17.473982 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b132c8b-6ec1-41c2-a614-b9a8251ab2ac None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:43:17.484607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b132c8b-6ec1-41c2-a614-b9a8251ab2ac None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:43:17.488087 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80132d15-4e0d-4510-a266-6409eaa35791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:43:17.488768 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80132d15-4e0d-4510-a266-6409eaa35791 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:43:17.489101 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80132d15-4e0d-4510-a266-6409eaa35791 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:43:17.489389 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80132d15-4e0d-4510-a266-6409eaa35791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:43:17.496353 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80132d15-4e0d-4510-a266-6409eaa35791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:43:17.497778 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80132d15-4e0d-4510-a266-6409eaa35791 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:43:17.505472 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1220/2410] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:43:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:43:17.586558 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f7111-2700-4b22-bef7-c12fee5903b8 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:43:17.586890 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f7111-2700-4b22-bef7-c12fee5903b8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:43:17.587173 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f7111-2700-4b22-bef7-c12fee5903b8 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:43:17.588565 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1191/2411] 10.222.0.67 () {54 vars in 921 bytes} [Tue May 31 14:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 31 14:43:17.594036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b674d-a55d-49ad-9822-ac35855bbaf5 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:43:17.594332 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b674d-a55d-49ad-9822-ac35855bbaf5 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:43:17.594618 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b674d-a55d-49ad-9822-ac35855bbaf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:43:17.613742 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6f0b674d-a55d-49ad-9822-ac35855bbaf5 None None] MFA Rules not processed for user `68c6363b60954c3ebf50d2019f5f2fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:43:17.614987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6f0b674d-a55d-49ad-9822-ac35855bbaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:43:17.624274 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1221/2412] 10.222.0.67 () {58 vars in 1018 bytes} [Tue May 31 14:43:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:43:17.634524 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f7111-2700-4b22-bef7-c12fee5903b8 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:43:17.636221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f45f7111-2700-4b22-bef7-c12fee5903b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:43:17.650116 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f7111-2700-4b22-bef7-c12fee5903b8 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:43:17.653866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664361c1-c4b2-4267-a56e-5a30a82d1fd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:43:17.654508 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664361c1-c4b2-4267-a56e-5a30a82d1fd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:43:17.654831 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664361c1-c4b2-4267-a56e-5a30a82d1fd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:43:17.655117 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664361c1-c4b2-4267-a56e-5a30a82d1fd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:43:17.665834 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664361c1-c4b2-4267-a56e-5a30a82d1fd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:43:17.667294 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664361c1-c4b2-4267-a56e-5a30a82d1fd7 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:43:17.678008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1192/2413] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:43:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:43:21.932215 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d27fe-822f-4ac9-81e8-533531f199f6 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:43:21.932739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d27fe-822f-4ac9-81e8-533531f199f6 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:43:21.932739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d27fe-822f-4ac9-81e8-533531f199f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:43:21.951827 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-658d27fe-822f-4ac9-81e8-533531f199f6 None None] MFA Rules not processed for user `0b7d100f2407432381d119c25814fff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:43:21.953067 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-658d27fe-822f-4ac9-81e8-533531f199f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:43:21.964266 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1222/2414] 10.222.0.67 () {58 vars in 1009 bytes} [Tue May 31 14:43:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:43:21.978204 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664361c1-c4b2-4267-a56e-5a30a82d1fd7 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:43:21.989376 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664361c1-c4b2-4267-a56e-5a30a82d1fd7 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:43:21.994343 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e8cf5-fb1b-4a1f-aa36-b74b9d4083e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfa2282f957941b98db1d7497da3351b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:43:21.995020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e8cf5-fb1b-4a1f-aa36-b74b9d4083e6 service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:43:21.995307 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e8cf5-fb1b-4a1f-aa36-b74b9d4083e6 service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:43:21.995585 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e8cf5-fb1b-4a1f-aa36-b74b9d4083e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:43:22.005257 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967e8cf5-fb1b-4a1f-aa36-b74b9d4083e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:43:22.007097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967e8cf5-fb1b-4a1f-aa36-b74b9d4083e6 service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:43:22.015451 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1193/2415] 10.222.0.67 () {60 vars in 1384 bytes} [Tue May 31 14:43:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 31 14:43:31.945720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=58 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:43:32.012139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=50 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:43:34.424716 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d27fe-822f-4ac9-81e8-533531f199f6 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:43:34.434992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d27fe-822f-4ac9-81e8-533531f199f6 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:43:34.438595 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef82a5e-20c1-499e-a203-9ee0846cfe09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:43:34.439271 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef82a5e-20c1-499e-a203-9ee0846cfe09 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:43:34.439557 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef82a5e-20c1-499e-a203-9ee0846cfe09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:43:34.439839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef82a5e-20c1-499e-a203-9ee0846cfe09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:43:34.441607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9ef82a5e-20c1-499e-a203-9ee0846cfe09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:43:34.454589 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef82a5e-20c1-499e-a203-9ee0846cfe09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:43:34.456048 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef82a5e-20c1-499e-a203-9ee0846cfe09 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:43:34.466958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1223/2416] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:43:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:43:34.864805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e8cf5-fb1b-4a1f-aa36-b74b9d4083e6 service swift] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:43:34.874009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e8cf5-fb1b-4a1f-aa36-b74b9d4083e6 service swift] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:43:34.877583 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31935ba4-1bee-4103-807c-a94959e3a79d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:43:34.878223 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31935ba4-1bee-4103-807c-a94959e3a79d service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:43:34.878511 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31935ba4-1bee-4103-807c-a94959e3a79d service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:43:34.878834 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31935ba4-1bee-4103-807c-a94959e3a79d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:43:34.880781 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-31935ba4-1bee-4103-807c-a94959e3a79d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:43:34.894323 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31935ba4-1bee-4103-807c-a94959e3a79d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:43:34.896276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31935ba4-1bee-4103-807c-a94959e3a79d service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:43:34.907324 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1194/2417] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:43:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:43:39.248879 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1f962-f4fc-4c67-a9ed-6069ed2ded25 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:43:39.249403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1f962-f4fc-4c67-a9ed-6069ed2ded25 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:43:39.249511 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1f962-f4fc-4c67-a9ed-6069ed2ded25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:43:39.267443 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-37d1f962-f4fc-4c67-a9ed-6069ed2ded25 None None] MFA Rules not processed for user `91255c0b2d914ef9be10eb69d7d8ed4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:43:39.268785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-37d1f962-f4fc-4c67-a9ed-6069ed2ded25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:43:39.322610 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1224/2418] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:43:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:43:43.279733 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31935ba4-1bee-4103-807c-a94959e3a79d service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:43:43.281183 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-31935ba4-1bee-4103-807c-a94959e3a79d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:43:43.295510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31935ba4-1bee-4103-807c-a94959e3a79d service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:43:43.301076 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff1ddd-e8fd-4999-aed5-5feb99e88f9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:43:43.302209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff1ddd-e8fd-4999-aed5-5feb99e88f9b service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:43:43.302495 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff1ddd-e8fd-4999-aed5-5feb99e88f9b service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:43:43.302809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff1ddd-e8fd-4999-aed5-5feb99e88f9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:43:43.304554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2aff1ddd-e8fd-4999-aed5-5feb99e88f9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:43:43.315195 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff1ddd-e8fd-4999-aed5-5feb99e88f9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:43:43.316757 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff1ddd-e8fd-4999-aed5-5feb99e88f9b service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:43:43.328214 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1195/2419] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:43:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:43:44.280155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d1f962-f4fc-4c67-a9ed-6069ed2ded25 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:43:44.281579 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-37d1f962-f4fc-4c67-a9ed-6069ed2ded25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:43:44.359630 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d1f962-f4fc-4c67-a9ed-6069ed2ded25 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:43:44.363166 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61afa6af-0a0d-4c53-96dd-87d5f5850436 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:43:44.363815 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61afa6af-0a0d-4c53-96dd-87d5f5850436 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:43:44.364102 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61afa6af-0a0d-4c53-96dd-87d5f5850436 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:43:44.364386 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61afa6af-0a0d-4c53-96dd-87d5f5850436 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:43:44.366690 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-61afa6af-0a0d-4c53-96dd-87d5f5850436 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:43:44.425475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61afa6af-0a0d-4c53-96dd-87d5f5850436 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:43:44.427630 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61afa6af-0a0d-4c53-96dd-87d5f5850436 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:43:44.440276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1225/2420] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:43:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:43:48.429785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a199203-03f9-4d6d-8c5b-4fb64cea6e90 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:43:48.430277 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a199203-03f9-4d6d-8c5b-4fb64cea6e90 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:43:48.430385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a199203-03f9-4d6d-8c5b-4fb64cea6e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:43:48.448810 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9a199203-03f9-4d6d-8c5b-4fb64cea6e90 None None] MFA Rules not processed for user `0b7d100f2407432381d119c25814fff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:43:48.449947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9a199203-03f9-4d6d-8c5b-4fb64cea6e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:43:48.457093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1196/2421] 10.222.0.67 () {58 vars in 967 bytes} [Tue May 31 14:43:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:43:48.468953 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61afa6af-0a0d-4c53-96dd-87d5f5850436 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:43:48.481167 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61afa6af-0a0d-4c53-96dd-87d5f5850436 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:43:48.486203 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381789a-e624-47ea-af5e-40c92cdd0ce0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfa2282f957941b98db1d7497da3351b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:43:48.486905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381789a-e624-47ea-af5e-40c92cdd0ce0 service swift] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:43:48.487195 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381789a-e624-47ea-af5e-40c92cdd0ce0 service swift] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:43:48.487474 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381789a-e624-47ea-af5e-40c92cdd0ce0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:43:48.496096 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381789a-e624-47ea-af5e-40c92cdd0ce0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:43:48.497675 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381789a-e624-47ea-af5e-40c92cdd0ce0 service swift] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:43:48.505702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1226/2422] 10.222.0.67 () {60 vars in 1384 bytes} [Tue May 31 14:43:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 31 14:43:58.443878 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=15 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:43:58.502492 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=94 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:44:03.423116 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a199203-03f9-4d6d-8c5b-4fb64cea6e90 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:44:03.433132 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a199203-03f9-4d6d-8c5b-4fb64cea6e90 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:44:03.437121 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1712bf-c454-4ad0-8a0d-03d0f4bdebac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:44:03.437881 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1712bf-c454-4ad0-8a0d-03d0f4bdebac service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:44:03.438167 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1712bf-c454-4ad0-8a0d-03d0f4bdebac service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:44:03.438441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1712bf-c454-4ad0-8a0d-03d0f4bdebac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:44:03.440305 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5a1712bf-c454-4ad0-8a0d-03d0f4bdebac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:44:03.450585 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1712bf-c454-4ad0-8a0d-03d0f4bdebac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:44:03.452042 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1712bf-c454-4ad0-8a0d-03d0f4bdebac service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:44:03.460170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1197/2423] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:44:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:44:13.456601 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=6 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:44:43.596893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381789a-e624-47ea-af5e-40c92cdd0ce0 service swift] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:44:43.598340 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6381789a-e624-47ea-af5e-40c92cdd0ce0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:44:43.614476 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381789a-e624-47ea-af5e-40c92cdd0ce0 service swift] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:44:43.620874 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62000ae2-fd81-4cb9-9fc2-b4796f8496e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:44:43.621430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62000ae2-fd81-4cb9-9fc2-b4796f8496e5 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:44:43.621776 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62000ae2-fd81-4cb9-9fc2-b4796f8496e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:44:43.622064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62000ae2-fd81-4cb9-9fc2-b4796f8496e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:44:43.623950 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-62000ae2-fd81-4cb9-9fc2-b4796f8496e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:44:43.637150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62000ae2-fd81-4cb9-9fc2-b4796f8496e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:44:43.638787 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62000ae2-fd81-4cb9-9fc2-b4796f8496e5 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:44:43.650578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1227/2424] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:44:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:44:45.150281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e1cba-6d9b-4acd-b1d2-8ff145af288b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:44:45.150782 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e1cba-6d9b-4acd-b1d2-8ff145af288b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:44:45.150973 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e1cba-6d9b-4acd-b1d2-8ff145af288b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:44:45.168752 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-287e1cba-6d9b-4acd-b1d2-8ff145af288b None None] MFA Rules not processed for user `95d4208cdd30408bae5bcdc4d9941cde`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:44:45.169958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-287e1cba-6d9b-4acd-b1d2-8ff145af288b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:44:45.177068 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1198/2425] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:44:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:44:45.490012 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62000ae2-fd81-4cb9-9fc2-b4796f8496e5 service nova] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:44:45.499334 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62000ae2-fd81-4cb9-9fc2-b4796f8496e5 service nova] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:44:45.502942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75a1bf-66dc-4fee-824f-e39dc62c92f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:44:45.503610 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75a1bf-66dc-4fee-824f-e39dc62c92f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:44:45.503897 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75a1bf-66dc-4fee-824f-e39dc62c92f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:44:45.504178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75a1bf-66dc-4fee-824f-e39dc62c92f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:44:45.505940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3c75a1bf-66dc-4fee-824f-e39dc62c92f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:44:45.515380 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75a1bf-66dc-4fee-824f-e39dc62c92f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:44:45.516813 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75a1bf-66dc-4fee-824f-e39dc62c92f4 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:44:45.525301 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1228/2426] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:44:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:44:45.998612 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287e1cba-6d9b-4acd-b1d2-8ff145af288b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:44:46.000051 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-287e1cba-6d9b-4acd-b1d2-8ff145af288b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:44:46.015693 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287e1cba-6d9b-4acd-b1d2-8ff145af288b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:44:46.020976 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d1060-2b5e-463e-bbdb-d21516bb5ddd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c1c13515a8b41a781478dc65e6a88b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:44:46.021641 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d1060-2b5e-463e-bbdb-d21516bb5ddd service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:44:46.021919 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d1060-2b5e-463e-bbdb-d21516bb5ddd service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:44:46.022194 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d1060-2b5e-463e-bbdb-d21516bb5ddd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:44:46.023998 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-432d1060-2b5e-463e-bbdb-d21516bb5ddd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:44:46.046237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d1060-2b5e-463e-bbdb-d21516bb5ddd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:44:46.046237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d1060-2b5e-463e-bbdb-d21516bb5ddd service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:44:46.054471 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1199/2427] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:44:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:44:55.521272 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=12 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:44:56.050809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=9 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:00.457234 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75a1bf-66dc-4fee-824f-e39dc62c92f4 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:00.472294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75a1bf-66dc-4fee-824f-e39dc62c92f4 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:00.478445 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246378f8-6480-4e80-b51b-1f73651a7328 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:00.479137 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246378f8-6480-4e80-b51b-1f73651a7328 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:00.479423 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246378f8-6480-4e80-b51b-1f73651a7328 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:00.479702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246378f8-6480-4e80-b51b-1f73651a7328 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:00.481686 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-246378f8-6480-4e80-b51b-1f73651a7328 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:00.494596 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246378f8-6480-4e80-b51b-1f73651a7328 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:00.496306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246378f8-6480-4e80-b51b-1f73651a7328 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:00.507976 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1229/2428] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:45:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:45:05.708513 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d1060-2b5e-463e-bbdb-d21516bb5ddd service placement] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:05.722445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d1060-2b5e-463e-bbdb-d21516bb5ddd service placement] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:05.729459 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48df644-a032-403e-8716-68f5e5ddd8e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:05.730134 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48df644-a032-403e-8716-68f5e5ddd8e6 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:05.730597 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48df644-a032-403e-8716-68f5e5ddd8e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:05.731083 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48df644-a032-403e-8716-68f5e5ddd8e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:05.733071 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b48df644-a032-403e-8716-68f5e5ddd8e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:05.747244 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48df644-a032-403e-8716-68f5e5ddd8e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:05.749145 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48df644-a032-403e-8716-68f5e5ddd8e6 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:05.769065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1200/2429] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:45:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:45:06.737653 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1bf264-c5e0-4f68-9859-3fd89e8bde2d None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:06.738197 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1bf264-c5e0-4f68-9859-3fd89e8bde2d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:06.738197 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1bf264-c5e0-4f68-9859-3fd89e8bde2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:06.759742 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-1c1bf264-c5e0-4f68-9859-3fd89e8bde2d None None] MFA Rules not processed for user `32e6ace174a840f78c8d570719c3f631`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:45:06.760995 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1c1bf264-c5e0-4f68-9859-3fd89e8bde2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:06.826328 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1230/2430] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:45:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:45:07.151812 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48df644-a032-403e-8716-68f5e5ddd8e6 service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:07.162453 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48df644-a032-403e-8716-68f5e5ddd8e6 service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:07.167545 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e0379-40cd-404d-8e5f-d7ec18e0dd46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:07.168399 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e0379-40cd-404d-8e5f-d7ec18e0dd46 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:07.168874 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e0379-40cd-404d-8e5f-d7ec18e0dd46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:07.169329 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e0379-40cd-404d-8e5f-d7ec18e0dd46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:07.174190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-692e0379-40cd-404d-8e5f-d7ec18e0dd46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:07.186796 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e0379-40cd-404d-8e5f-d7ec18e0dd46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:07.188646 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e0379-40cd-404d-8e5f-d7ec18e0dd46 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:07.198585 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1201/2431] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:45:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:45:08.375550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1bf264-c5e0-4f68-9859-3fd89e8bde2d None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:08.385333 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1bf264-c5e0-4f68-9859-3fd89e8bde2d None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:08.388956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c21a34-c6cb-4dbb-a6eb-a6d9ab72f8ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:08.389600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c21a34-c6cb-4dbb-a6eb-a6d9ab72f8ce service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:08.389886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c21a34-c6cb-4dbb-a6eb-a6d9ab72f8ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:08.390165 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c21a34-c6cb-4dbb-a6eb-a6d9ab72f8ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:08.391984 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-38c21a34-c6cb-4dbb-a6eb-a6d9ab72f8ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:08.456609 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c21a34-c6cb-4dbb-a6eb-a6d9ab72f8ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:08.458018 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c21a34-c6cb-4dbb-a6eb-a6d9ab72f8ce service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:08.468591 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1231/2432] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:45:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:45:14.375028 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e0379-40cd-404d-8e5f-d7ec18e0dd46 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:14.387686 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e0379-40cd-404d-8e5f-d7ec18e0dd46 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:14.392786 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642bb2a-db18-4d4d-983b-62ed33171473 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c1c13515a8b41a781478dc65e6a88b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:14.393463 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642bb2a-db18-4d4d-983b-62ed33171473 service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:14.393743 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642bb2a-db18-4d4d-983b-62ed33171473 service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:14.394231 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642bb2a-db18-4d4d-983b-62ed33171473 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:14.396260 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7642bb2a-db18-4d4d-983b-62ed33171473 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:14.408898 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642bb2a-db18-4d4d-983b-62ed33171473 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:14.410807 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642bb2a-db18-4d4d-983b-62ed33171473 service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:14.423105 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1202/2433] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:45:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:45:14.436201 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c21a34-c6cb-4dbb-a6eb-a6d9ab72f8ce service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:14.439036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-38c21a34-c6cb-4dbb-a6eb-a6d9ab72f8ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:14.459393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c21a34-c6cb-4dbb-a6eb-a6d9ab72f8ce service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:14.464751 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aeadbd-e51a-4aff-bc2e-36a5c865dcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:14.466020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aeadbd-e51a-4aff-bc2e-36a5c865dcd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:14.466391 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aeadbd-e51a-4aff-bc2e-36a5c865dcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:14.466793 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aeadbd-e51a-4aff-bc2e-36a5c865dcd3 admin admin] PATH_INFO: `/v3/users/91255c0b2d914ef9be10eb69d7d8ed4f` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:14.468133 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aeadbd-e51a-4aff-bc2e-36a5c865dcd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91255c0b2d914ef9be10eb69d7d8ed4f)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:14.470150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aeadbd-e51a-4aff-bc2e-36a5c865dcd3 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:14.519982 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-25aeadbd-e51a-4aff-bc2e-36a5c865dcd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91255c0b2d914ef9be10eb69d7d8ed4f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:14.527418 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-25aeadbd-e51a-4aff-bc2e-36a5c865dcd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91255c0b2d914ef9be10eb69d7d8ed4f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:14.528037 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-25aeadbd-e51a-4aff-bc2e-36a5c865dcd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91255c0b2d914ef9be10eb69d7d8ed4f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:14.540514 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-25aeadbd-e51a-4aff-bc2e-36a5c865dcd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91255c0b2d914ef9be10eb69d7d8ed4f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:14.541218 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-25aeadbd-e51a-4aff-bc2e-36a5c865dcd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91255c0b2d914ef9be10eb69d7d8ed4f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:14.549477 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:14.556268 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1232/2434] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:45:14 2022] DELETE /identity/v3/users/91255c0b2d914ef9be10eb69d7d8ed4f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:45:14.746665 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642bb2a-db18-4d4d-983b-62ed33171473 service placement] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:14.759986 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642bb2a-db18-4d4d-983b-62ed33171473 service placement] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:14.818144 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07045c-352c-4bfc-a026-93ee1094d2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:14.819040 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07045c-352c-4bfc-a026-93ee1094d2bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:14.819323 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07045c-352c-4bfc-a026-93ee1094d2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:14.819597 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07045c-352c-4bfc-a026-93ee1094d2bd admin admin] PATH_INFO: `/v3/projects/8c940479079b46cf82136117f62c3f1a` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:14.820799 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07045c-352c-4bfc-a026-93ee1094d2bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c940479079b46cf82136117f62c3f1a)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:14.822730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07045c-352c-4bfc-a026-93ee1094d2bd admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:14.873378 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dc07045c-352c-4bfc-a026-93ee1094d2bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c940479079b46cf82136117f62c3f1a', 'request_id': 'req-dc07045c-352c-4bfc-a026-93ee1094d2bd'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:14.883146 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dc07045c-352c-4bfc-a026-93ee1094d2bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c940479079b46cf82136117f62c3f1a', 'request_id': 'req-dc07045c-352c-4bfc-a026-93ee1094d2bd'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:14.889856 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dc07045c-352c-4bfc-a026-93ee1094d2bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c940479079b46cf82136117f62c3f1a', 'request_id': 'req-dc07045c-352c-4bfc-a026-93ee1094d2bd'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:14.905321 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dc07045c-352c-4bfc-a026-93ee1094d2bd admin admin] The token cache is being invalidate because project 8c940479079b46cf82136117f62c3f1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:45:14.905615 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dc07045c-352c-4bfc-a026-93ee1094d2bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:14.906271 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dc07045c-352c-4bfc-a026-93ee1094d2bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:14.908119 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1203/2435] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:45:14 2022] DELETE /identity/v3/projects/8c940479079b46cf82136117f62c3f1a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:45:14.912915 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa223c-6426-4c97-8744-c364bc2a7f48 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:14.913203 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa223c-6426-4c97-8744-c364bc2a7f48 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:14.913483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa223c-6426-4c97-8744-c364bc2a7f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:14.942857 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-bbfa223c-6426-4c97-8744-c364bc2a7f48 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:45:14.944084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-bbfa223c-6426-4c97-8744-c364bc2a7f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:15.012665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1233/2436] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:45:15.020583 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe943f-b269-4c37-a74b-668e621c30d7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:15.020903 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe943f-b269-4c37-a74b-668e621c30d7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:15.021210 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe943f-b269-4c37-a74b-668e621c30d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:15.041207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3efe943f-b269-4c37-a74b-668e621c30d7 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:45:15.042430 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3efe943f-b269-4c37-a74b-668e621c30d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:15.050441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1204/2437] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:45:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:45:15.054925 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa223c-6426-4c97-8744-c364bc2a7f48 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:15.065348 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa223c-6426-4c97-8744-c364bc2a7f48 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:15.070766 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc336bc5-2e58-4655-b86b-dd798769120b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:15.071428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc336bc5-2e58-4655-b86b-dd798769120b admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:15.071712 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc336bc5-2e58-4655-b86b-dd798769120b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:15.072090 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc336bc5-2e58-4655-b86b-dd798769120b admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:15.072708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc336bc5-2e58-4655-b86b-dd798769120b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:45:15.073015 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc336bc5-2e58-4655-b86b-dd798769120b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:15.074599 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc336bc5-2e58-4655-b86b-dd798769120b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:15.087313 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1234/2438] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:45:15.091704 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe943f-b269-4c37-a74b-668e621c30d7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:15.100952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe943f-b269-4c37-a74b-668e621c30d7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:15.108265 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f6c03-e268-461b-9f1b-db262b82abba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:15.108992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f6c03-e268-461b-9f1b-db262b82abba admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:15.109330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f6c03-e268-461b-9f1b-db262b82abba admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:15.109811 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f6c03-e268-461b-9f1b-db262b82abba admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:15.110935 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78f6c03-e268-461b-9f1b-db262b82abba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:15.112828 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78f6c03-e268-461b-9f1b-db262b82abba admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:15.148940 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1205/2439] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:45:15 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:45:15.153144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc336bc5-2e58-4655-b86b-dd798769120b admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:15.163065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc336bc5-2e58-4655-b86b-dd798769120b admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:15.218796 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cf8d2-3845-4ce1-b310-ed2c3b0faa0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:15.219503 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cf8d2-3845-4ce1-b310-ed2c3b0faa0e admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:15.219793 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cf8d2-3845-4ce1-b310-ed2c3b0faa0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:15.220073 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cf8d2-3845-4ce1-b310-ed2c3b0faa0e admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:15.220798 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050cf8d2-3845-4ce1-b310-ed2c3b0faa0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:15.222759 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050cf8d2-3845-4ce1-b310-ed2c3b0faa0e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:15.244118 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1235/2440] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:45:15 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:45:15.248180 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f6c03-e268-461b-9f1b-db262b82abba admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:15.258619 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f6c03-e268-461b-9f1b-db262b82abba admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:15.263354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6a68c-6338-4c15-b3fb-06c87971880f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:15.264016 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6a68c-6338-4c15-b3fb-06c87971880f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:15.264299 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6a68c-6338-4c15-b3fb-06c87971880f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:15.264577 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6a68c-6338-4c15-b3fb-06c87971880f admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:15.265304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b6a68c-6338-4c15-b3fb-06c87971880f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:15.267017 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b6a68c-6338-4c15-b3fb-06c87971880f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:15.277273 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1206/2441] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:45:15.281511 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cf8d2-3845-4ce1-b310-ed2c3b0faa0e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:15.292399 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cf8d2-3845-4ce1-b310-ed2c3b0faa0e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:15.297219 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80223df6-1f28-42c0-9609-1471e2db7466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:15.297885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80223df6-1f28-42c0-9609-1471e2db7466 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:15.298175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80223df6-1f28-42c0-9609-1471e2db7466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:15.298455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80223df6-1f28-42c0-9609-1471e2db7466 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:15.299238 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80223df6-1f28-42c0-9609-1471e2db7466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:15.300837 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80223df6-1f28-42c0-9609-1471e2db7466 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:15.312026 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1236/2442] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:45:15.316179 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6a68c-6338-4c15-b3fb-06c87971880f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:15.321520 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:15.327908 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6a68c-6338-4c15-b3fb-06c87971880f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:15.333528 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d65190-fe3c-465d-bbf0-f1c7494465bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:15.334426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d65190-fe3c-465d-bbf0-f1c7494465bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:15.334745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d65190-fe3c-465d-bbf0-f1c7494465bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:15.335023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d65190-fe3c-465d-bbf0-f1c7494465bd admin admin] PATH_INFO: `/v3/projects/5a6f3e70f8874b7395872dd1f53af1dc/users/8a149ba95adb4684bd762cbda6bb0b6c/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:15.350604 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d65190-fe3c-465d-bbf0-f1c7494465bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a6f3e70f8874b7395872dd1f53af1dc, user_id=8a149ba95adb4684bd762cbda6bb0b6c, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:15.353905 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d65190-fe3c-465d-bbf0-f1c7494465bd admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:15.368891 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1207/2443] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:45:15 2022] PUT /identity/v3/projects/5a6f3e70f8874b7395872dd1f53af1dc/users/8a149ba95adb4684bd762cbda6bb0b6c/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:45:15.380057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb1550-f5d2-44d3-858b-5d26a51783b3 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:15.380350 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb1550-f5d2-44d3-858b-5d26a51783b3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:15.380659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb1550-f5d2-44d3-858b-5d26a51783b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:15.413024 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-0dbb1550-f5d2-44d3-858b-5d26a51783b3 None None] MFA Rules not processed for user `8a149ba95adb4684bd762cbda6bb0b6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:45:15.414840 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0dbb1550-f5d2-44d3-858b-5d26a51783b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:15.468891 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:15.480506 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1237/2444] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:45:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:45:15.484933 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d65190-fe3c-465d-bbf0-f1c7494465bd admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:15.549876 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d65190-fe3c-465d-bbf0-f1c7494465bd admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:15.556030 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02653004-57ca-4081-a61d-791972df2646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:15.556778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02653004-57ca-4081-a61d-791972df2646 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:15.557068 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02653004-57ca-4081-a61d-791972df2646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:15.557703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02653004-57ca-4081-a61d-791972df2646 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:15.558681 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02653004-57ca-4081-a61d-791972df2646 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:15.560465 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02653004-57ca-4081-a61d-791972df2646 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:15.591699 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1208/2445] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:45:15 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:45:15.592893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb1550-f5d2-44d3-858b-5d26a51783b3 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:15.607908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb1550-f5d2-44d3-858b-5d26a51783b3 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:15.696951 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deafdfec-ae40-4ad8-a470-a3d9a6bca097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:15.697717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deafdfec-ae40-4ad8-a470-a3d9a6bca097 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:15.698157 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deafdfec-ae40-4ad8-a470-a3d9a6bca097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:15.698791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deafdfec-ae40-4ad8-a470-a3d9a6bca097 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:15.699637 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deafdfec-ae40-4ad8-a470-a3d9a6bca097 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:15.701943 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deafdfec-ae40-4ad8-a470-a3d9a6bca097 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:15.730054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1238/2446] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:45:15 2022] POST /identity/v3/users => generated 297 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:45:15.734038 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02653004-57ca-4081-a61d-791972df2646 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:15.745096 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02653004-57ca-4081-a61d-791972df2646 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:15.749782 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6bca7-cc02-48e4-9739-5ad39b63d359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:15.750440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6bca7-cc02-48e4-9739-5ad39b63d359 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:15.750752 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6bca7-cc02-48e4-9739-5ad39b63d359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:15.751030 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6bca7-cc02-48e4-9739-5ad39b63d359 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:15.751749 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6bca7-cc02-48e4-9739-5ad39b63d359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:15.753332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6bca7-cc02-48e4-9739-5ad39b63d359 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:15.764515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1209/2447] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:45:15.768868 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deafdfec-ae40-4ad8-a470-a3d9a6bca097 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:15.779121 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deafdfec-ae40-4ad8-a470-a3d9a6bca097 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:15.783888 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824abb8-1c7f-4e9f-a184-904f83bd8a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:15.784782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824abb8-1c7f-4e9f-a184-904f83bd8a51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:15.785067 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824abb8-1c7f-4e9f-a184-904f83bd8a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:15.785342 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824abb8-1c7f-4e9f-a184-904f83bd8a51 admin admin] PATH_INFO: `/v3/projects/16da423d3fff40e4bd188b1869c084a2/users/9d854416104e4f86b627696bbc9e9938/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:15.801194 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824abb8-1c7f-4e9f-a184-904f83bd8a51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16da423d3fff40e4bd188b1869c084a2, user_id=9d854416104e4f86b627696bbc9e9938, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:15.803800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824abb8-1c7f-4e9f-a184-904f83bd8a51 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:15.818047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1239/2448] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:45:15 2022] PUT /identity/v3/projects/16da423d3fff40e4bd188b1869c084a2/users/9d854416104e4f86b627696bbc9e9938/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:45:15.822357 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6bca7-cc02-48e4-9739-5ad39b63d359 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:15.832408 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6bca7-cc02-48e4-9739-5ad39b63d359 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:15.890239 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07edaec2-7526-4d89-bf25-860f92a49023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:15.890956 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edaec2-7526-4d89-bf25-860f92a49023 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:15.891242 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edaec2-7526-4d89-bf25-860f92a49023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:15.891520 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edaec2-7526-4d89-bf25-860f92a49023 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:15.892263 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edaec2-7526-4d89-bf25-860f92a49023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:15.893888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edaec2-7526-4d89-bf25-860f92a49023 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:15.904630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1210/2449] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:45:15.909201 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824abb8-1c7f-4e9f-a184-904f83bd8a51 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:15.920634 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824abb8-1c7f-4e9f-a184-904f83bd8a51 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:15.925537 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ce117-8f6d-46d3-9505-b453a23b2e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:15.926485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ce117-8f6d-46d3-9505-b453a23b2e6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:15.927194 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ce117-8f6d-46d3-9505-b453a23b2e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:15.927480 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ce117-8f6d-46d3-9505-b453a23b2e6f admin admin] PATH_INFO: `/v3/projects/16da423d3fff40e4bd188b1869c084a2/users/9d854416104e4f86b627696bbc9e9938/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:15.930056 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ce117-8f6d-46d3-9505-b453a23b2e6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=16da423d3fff40e4bd188b1869c084a2, user_id=9d854416104e4f86b627696bbc9e9938, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:15.933153 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ce117-8f6d-46d3-9505-b453a23b2e6f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:15.945665 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1240/2450] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:45:15 2022] PUT /identity/v3/projects/16da423d3fff40e4bd188b1869c084a2/users/9d854416104e4f86b627696bbc9e9938/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:45:15.958631 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0da08-3234-4639-93d0-0882189eb279 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:15.958970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0da08-3234-4639-93d0-0882189eb279 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:15.959248 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0da08-3234-4639-93d0-0882189eb279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:15.997828 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e8e0da08-3234-4639-93d0-0882189eb279 None None] MFA Rules not processed for user `9d854416104e4f86b627696bbc9e9938`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:45:16.000791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e8e0da08-3234-4639-93d0-0882189eb279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:16.050541 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:16.090863 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1211/2451] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:45:15 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:45:16.102911 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ce117-8f6d-46d3-9505-b453a23b2e6f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:16.105350 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d57ce117-8f6d-46d3-9505-b453a23b2e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:16.189957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ce117-8f6d-46d3-9505-b453a23b2e6f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:16.193879 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e6976-7082-4da2-9907-0001a6cbc3c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:16.194847 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e6976-7082-4da2-9907-0001a6cbc3c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:16.195345 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e6976-7082-4da2-9907-0001a6cbc3c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:16.195690 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e6976-7082-4da2-9907-0001a6cbc3c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:16.206075 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e6976-7082-4da2-9907-0001a6cbc3c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:16.207709 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e6976-7082-4da2-9907-0001a6cbc3c1 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:16.221591 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1241/2452] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:45:16 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:45:16.303354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0da08-3234-4639-93d0-0882189eb279 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:16.313665 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0da08-3234-4639-93d0-0882189eb279 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:16.319303 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3715286-d155-4508-a309-91297d8c811a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:16.320017 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3715286-d155-4508-a309-91297d8c811a service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:16.320304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3715286-d155-4508-a309-91297d8c811a service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:16.320579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3715286-d155-4508-a309-91297d8c811a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:16.381267 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3715286-d155-4508-a309-91297d8c811a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:16.382736 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3715286-d155-4508-a309-91297d8c811a service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:16.391781 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1212/2453] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:45:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:45:18.490027 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e6976-7082-4da2-9907-0001a6cbc3c1 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:18.491694 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-711e6976-7082-4da2-9907-0001a6cbc3c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:18.551029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:18.581155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e6976-7082-4da2-9907-0001a6cbc3c1 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:18.587543 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9116d16-c97f-4c69-b686-6bdfecb406ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:18.588216 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9116d16-c97f-4c69-b686-6bdfecb406ed service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:18.588508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9116d16-c97f-4c69-b686-6bdfecb406ed service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:18.588792 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9116d16-c97f-4c69-b686-6bdfecb406ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:18.591912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b9116d16-c97f-4c69-b686-6bdfecb406ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:18.660897 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9116d16-c97f-4c69-b686-6bdfecb406ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:18.662755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9116d16-c97f-4c69-b686-6bdfecb406ed service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:18.673910 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1242/2454] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:45:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:45:19.395268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91667aab-8c5d-45ed-beb1-fd7296b1e048 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:19.395793 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91667aab-8c5d-45ed-beb1-fd7296b1e048 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:19.395926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91667aab-8c5d-45ed-beb1-fd7296b1e048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:19.413225 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-91667aab-8c5d-45ed-beb1-fd7296b1e048 None None] MFA Rules not processed for user `a36fb9afb5d54d3493e8752bae88dd13`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:45:19.415023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-91667aab-8c5d-45ed-beb1-fd7296b1e048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:19.469090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1213/2455] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:45:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:45:27.454023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9116d16-c97f-4c69-b686-6bdfecb406ed service nova] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:27.520531 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9116d16-c97f-4c69-b686-6bdfecb406ed service nova] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:27.525778 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e1317-fb4e-4ec7-accb-285d15f0f4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:27.526697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e1317-fb4e-4ec7-accb-285d15f0f4b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:27.526988 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e1317-fb4e-4ec7-accb-285d15f0f4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:27.527270 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e1317-fb4e-4ec7-accb-285d15f0f4b9 admin admin] PATH_INFO: `/v3/users/8a149ba95adb4684bd762cbda6bb0b6c` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:27.528488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e1317-fb4e-4ec7-accb-285d15f0f4b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a149ba95adb4684bd762cbda6bb0b6c)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:27.531105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e1317-fb4e-4ec7-accb-285d15f0f4b9 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:27.579132 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a97e1317-fb4e-4ec7-accb-285d15f0f4b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a149ba95adb4684bd762cbda6bb0b6c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:27.588193 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a97e1317-fb4e-4ec7-accb-285d15f0f4b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a149ba95adb4684bd762cbda6bb0b6c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:27.588907 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a97e1317-fb4e-4ec7-accb-285d15f0f4b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a149ba95adb4684bd762cbda6bb0b6c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:27.609044 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a97e1317-fb4e-4ec7-accb-285d15f0f4b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a149ba95adb4684bd762cbda6bb0b6c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:27.609778 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a97e1317-fb4e-4ec7-accb-285d15f0f4b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a149ba95adb4684bd762cbda6bb0b6c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:27.623145 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1243/2456] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:45:27 2022] DELETE /identity/v3/users/8a149ba95adb4684bd762cbda6bb0b6c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:45:27.627887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d89f8-5d3a-4fb3-a8bf-7dc2b4849ebf None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:27.628388 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d89f8-5d3a-4fb3-a8bf-7dc2b4849ebf None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:27.628586 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d89f8-5d3a-4fb3-a8bf-7dc2b4849ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:27.649837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-265d89f8-5d3a-4fb3-a8bf-7dc2b4849ebf None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:45:27.660276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-265d89f8-5d3a-4fb3-a8bf-7dc2b4849ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:27.723374 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1214/2457] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 14:45:27.733766 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e1317-fb4e-4ec7-accb-285d15f0f4b9 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:27.735219 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a97e1317-fb4e-4ec7-accb-285d15f0f4b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:27.741844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:27.796644 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e1317-fb4e-4ec7-accb-285d15f0f4b9 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:27.801106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ce252-73ae-4db4-86cc-0f5b3b15050e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:27.801777 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ce252-73ae-4db4-86cc-0f5b3b15050e service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:27.802063 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ce252-73ae-4db4-86cc-0f5b3b15050e service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:27.802345 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ce252-73ae-4db4-86cc-0f5b3b15050e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:27.813117 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ce252-73ae-4db4-86cc-0f5b3b15050e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:27.814584 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ce252-73ae-4db4-86cc-0f5b3b15050e service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:27.825711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1244/2458] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:45:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:45:28.149613 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d89f8-5d3a-4fb3-a8bf-7dc2b4849ebf None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:28.151246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-265d89f8-5d3a-4fb3-a8bf-7dc2b4849ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:28.193296 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d89f8-5d3a-4fb3-a8bf-7dc2b4849ebf None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:28.204011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d7f1c-9e34-475b-a2f7-466671fbb7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:28.204921 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d7f1c-9e34-475b-a2f7-466671fbb7ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:28.205196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d7f1c-9e34-475b-a2f7-466671fbb7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:28.205476 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d7f1c-9e34-475b-a2f7-466671fbb7ea admin admin] PATH_INFO: `/v3/projects/5a6f3e70f8874b7395872dd1f53af1dc` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:28.206749 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d7f1c-9e34-475b-a2f7-466671fbb7ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a6f3e70f8874b7395872dd1f53af1dc)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:28.208679 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d7f1c-9e34-475b-a2f7-466671fbb7ea admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:28.222202 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:28.260464 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-885d7f1c-9e34-475b-a2f7-466671fbb7ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a6f3e70f8874b7395872dd1f53af1dc', 'request_id': 'req-885d7f1c-9e34-475b-a2f7-466671fbb7ea'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:28.266862 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-885d7f1c-9e34-475b-a2f7-466671fbb7ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a6f3e70f8874b7395872dd1f53af1dc', 'request_id': 'req-885d7f1c-9e34-475b-a2f7-466671fbb7ea'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:28.273998 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-885d7f1c-9e34-475b-a2f7-466671fbb7ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a6f3e70f8874b7395872dd1f53af1dc', 'request_id': 'req-885d7f1c-9e34-475b-a2f7-466671fbb7ea'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:28.289738 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-885d7f1c-9e34-475b-a2f7-466671fbb7ea admin admin] The token cache is being invalidate because project 5a6f3e70f8874b7395872dd1f53af1dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:45:28.290056 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-885d7f1c-9e34-475b-a2f7-466671fbb7ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:28.290676 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-885d7f1c-9e34-475b-a2f7-466671fbb7ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:28.292275 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1215/2459] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:45:28 2022] DELETE /identity/v3/projects/5a6f3e70f8874b7395872dd1f53af1dc => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:45:28.296385 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ce252-73ae-4db4-86cc-0f5b3b15050e service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:28.306791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ce252-73ae-4db4-86cc-0f5b3b15050e service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:28.365561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cded5e-179e-4cf9-a312-10791ca0b8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:28.366416 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cded5e-179e-4cf9-a312-10791ca0b8f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:28.366729 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cded5e-179e-4cf9-a312-10791ca0b8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:28.367007 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cded5e-179e-4cf9-a312-10791ca0b8f8 admin admin] PATH_INFO: `/v3/users/9d854416104e4f86b627696bbc9e9938` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:28.368249 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cded5e-179e-4cf9-a312-10791ca0b8f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d854416104e4f86b627696bbc9e9938)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:28.370190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cded5e-179e-4cf9-a312-10791ca0b8f8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:28.421869 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-80cded5e-179e-4cf9-a312-10791ca0b8f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d854416104e4f86b627696bbc9e9938'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:28.430216 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-80cded5e-179e-4cf9-a312-10791ca0b8f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d854416104e4f86b627696bbc9e9938'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:28.430917 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-80cded5e-179e-4cf9-a312-10791ca0b8f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d854416104e4f86b627696bbc9e9938'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:28.443331 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-80cded5e-179e-4cf9-a312-10791ca0b8f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d854416104e4f86b627696bbc9e9938'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:28.444020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-80cded5e-179e-4cf9-a312-10791ca0b8f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d854416104e4f86b627696bbc9e9938'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:28.458321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1245/2460] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:45:28 2022] DELETE /identity/v3/users/9d854416104e4f86b627696bbc9e9938 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:45:28.754865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d7f1c-9e34-475b-a2f7-466671fbb7ea admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:28.756505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-885d7f1c-9e34-475b-a2f7-466671fbb7ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:28.777522 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d7f1c-9e34-475b-a2f7-466671fbb7ea admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:28.846601 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0226c-2009-4811-81d5-3bb2a2985d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:28.847729 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0226c-2009-4811-81d5-3bb2a2985d92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:28.848040 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0226c-2009-4811-81d5-3bb2a2985d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:28.848372 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0226c-2009-4811-81d5-3bb2a2985d92 admin admin] PATH_INFO: `/v3/projects/16da423d3fff40e4bd188b1869c084a2` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:28.849857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d0226c-2009-4811-81d5-3bb2a2985d92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16da423d3fff40e4bd188b1869c084a2)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:28.852353 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d0226c-2009-4811-81d5-3bb2a2985d92 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:28.904447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-75d0226c-2009-4811-81d5-3bb2a2985d92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16da423d3fff40e4bd188b1869c084a2', 'request_id': 'req-75d0226c-2009-4811-81d5-3bb2a2985d92'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:28.910540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-75d0226c-2009-4811-81d5-3bb2a2985d92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16da423d3fff40e4bd188b1869c084a2', 'request_id': 'req-75d0226c-2009-4811-81d5-3bb2a2985d92'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:28.917057 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-75d0226c-2009-4811-81d5-3bb2a2985d92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16da423d3fff40e4bd188b1869c084a2', 'request_id': 'req-75d0226c-2009-4811-81d5-3bb2a2985d92'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:28.933695 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-75d0226c-2009-4811-81d5-3bb2a2985d92 admin admin] The token cache is being invalidate because project 16da423d3fff40e4bd188b1869c084a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:45:28.934007 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-75d0226c-2009-4811-81d5-3bb2a2985d92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:28.934686 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-75d0226c-2009-4811-81d5-3bb2a2985d92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:28.936237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1216/2461] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:45:28 2022] DELETE /identity/v3/projects/16da423d3fff40e4bd188b1869c084a2 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:45:28.941291 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d786970-bc65-479d-acf0-c94a64301b64 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:28.941583 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d786970-bc65-479d-acf0-c94a64301b64 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:28.941866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d786970-bc65-479d-acf0-c94a64301b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:28.959873 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-4d786970-bc65-479d-acf0-c94a64301b64 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:45:28.961130 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4d786970-bc65-479d-acf0-c94a64301b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:28.982206 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:29.023809 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1246/2462] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:45:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:45:29.032967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65637110-107a-4354-9530-582ef2517cfa None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:29.033264 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65637110-107a-4354-9530-582ef2517cfa None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:29.033549 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65637110-107a-4354-9530-582ef2517cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:29.051640 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-65637110-107a-4354-9530-582ef2517cfa None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:45:29.052858 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-65637110-107a-4354-9530-582ef2517cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:29.061640 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1217/2463] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:45:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:45:29.065873 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d786970-bc65-479d-acf0-c94a64301b64 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:29.076249 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d786970-bc65-479d-acf0-c94a64301b64 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:29.080777 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03784f3-fb59-4e67-89c1-401bf6ba744c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:29.081428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03784f3-fb59-4e67-89c1-401bf6ba744c admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:29.081714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03784f3-fb59-4e67-89c1-401bf6ba744c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:29.081989 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03784f3-fb59-4e67-89c1-401bf6ba744c admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:29.082750 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03784f3-fb59-4e67-89c1-401bf6ba744c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:45:29.083060 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03784f3-fb59-4e67-89c1-401bf6ba744c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:29.084654 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03784f3-fb59-4e67-89c1-401bf6ba744c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:29.096678 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1247/2464] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:45:29.101102 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65637110-107a-4354-9530-582ef2517cfa None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:29.110496 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65637110-107a-4354-9530-582ef2517cfa None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:29.115194 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c988c-6301-483f-95fa-a59603c590b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:29.115860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c988c-6301-483f-95fa-a59603c590b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:29.116711 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c988c-6301-483f-95fa-a59603c590b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:29.116953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c988c-6301-483f-95fa-a59603c590b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:29.117660 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c988c-6301-483f-95fa-a59603c590b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:29.119615 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c988c-6301-483f-95fa-a59603c590b2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:29.155138 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1218/2465] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:45:29 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:45:29.159561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03784f3-fb59-4e67-89c1-401bf6ba744c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:29.170081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03784f3-fb59-4e67-89c1-401bf6ba744c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:29.238117 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7dd4b9-2538-4e38-8923-97de5f871601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:29.238921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7dd4b9-2538-4e38-8923-97de5f871601 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:29.239269 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7dd4b9-2538-4e38-8923-97de5f871601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:29.239612 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7dd4b9-2538-4e38-8923-97de5f871601 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:29.240432 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7dd4b9-2538-4e38-8923-97de5f871601 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:29.243017 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7dd4b9-2538-4e38-8923-97de5f871601 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:29.284515 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1248/2466] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:45:29 2022] POST /identity/v3/users => generated 294 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:45:29.284768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c988c-6301-483f-95fa-a59603c590b2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:29.292740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:29.302145 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c988c-6301-483f-95fa-a59603c590b2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:29.310153 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77365ff4-ea31-4852-a8a6-141fcc0b1477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:29.311897 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77365ff4-ea31-4852-a8a6-141fcc0b1477 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:29.311897 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77365ff4-ea31-4852-a8a6-141fcc0b1477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:29.312124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77365ff4-ea31-4852-a8a6-141fcc0b1477 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:29.312750 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77365ff4-ea31-4852-a8a6-141fcc0b1477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:29.314526 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77365ff4-ea31-4852-a8a6-141fcc0b1477 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:29.327249 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1219/2467] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:45:29.331498 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7dd4b9-2538-4e38-8923-97de5f871601 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:29.342368 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7dd4b9-2538-4e38-8923-97de5f871601 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:29.347310 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81dff50-3362-4c19-ae6b-aa9d5f3b0cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:29.348185 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81dff50-3362-4c19-ae6b-aa9d5f3b0cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:29.348320 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81dff50-3362-4c19-ae6b-aa9d5f3b0cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:29.348532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81dff50-3362-4c19-ae6b-aa9d5f3b0cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:29.349268 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81dff50-3362-4c19-ae6b-aa9d5f3b0cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:29.351680 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81dff50-3362-4c19-ae6b-aa9d5f3b0cc7 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:29.367670 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1249/2468] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:45:29.371965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77365ff4-ea31-4852-a8a6-141fcc0b1477 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:29.384239 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77365ff4-ea31-4852-a8a6-141fcc0b1477 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:29.390179 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294210ef-f9ff-413b-b48a-687be998a1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:29.391120 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294210ef-f9ff-413b-b48a-687be998a1c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:29.391475 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294210ef-f9ff-413b-b48a-687be998a1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:29.391700 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294210ef-f9ff-413b-b48a-687be998a1c4 admin admin] PATH_INFO: `/v3/projects/53a056f221f84fd3a165dcb180033de6/users/d114e710cdd44988b07b166d30ef318c/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:29.407565 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294210ef-f9ff-413b-b48a-687be998a1c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53a056f221f84fd3a165dcb180033de6, user_id=d114e710cdd44988b07b166d30ef318c, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:29.410120 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294210ef-f9ff-413b-b48a-687be998a1c4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:29.428318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1220/2469] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:45:29 2022] PUT /identity/v3/projects/53a056f221f84fd3a165dcb180033de6/users/d114e710cdd44988b07b166d30ef318c/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:45:29.436043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3713b0e-8cde-46a2-9742-6ccded861569 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:29.436331 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3713b0e-8cde-46a2-9742-6ccded861569 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:29.436611 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3713b0e-8cde-46a2-9742-6ccded861569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:29.464278 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-b3713b0e-8cde-46a2-9742-6ccded861569 None None] MFA Rules not processed for user `d114e710cdd44988b07b166d30ef318c`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:45:29.465527 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b3713b0e-8cde-46a2-9742-6ccded861569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:29.541636 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1250/2470] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:45:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:45:29.546338 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294210ef-f9ff-413b-b48a-687be998a1c4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:29.613886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294210ef-f9ff-413b-b48a-687be998a1c4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:29.619178 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267bb2f-e55a-48c6-a754-587c20e4459d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:29.619859 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267bb2f-e55a-48c6-a754-587c20e4459d admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:29.620151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267bb2f-e55a-48c6-a754-587c20e4459d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:29.620435 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267bb2f-e55a-48c6-a754-587c20e4459d admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:29.621137 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0267bb2f-e55a-48c6-a754-587c20e4459d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:29.623096 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0267bb2f-e55a-48c6-a754-587c20e4459d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:29.653082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1221/2471] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:45:29 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:45:29.654191 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3713b0e-8cde-46a2-9742-6ccded861569 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:29.665423 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3713b0e-8cde-46a2-9742-6ccded861569 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:29.685188 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:29.730982 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e8ca4-a174-4953-88f6-c4c8b3432c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:29.731697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e8ca4-a174-4953-88f6-c4c8b3432c0f admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:29.732005 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e8ca4-a174-4953-88f6-c4c8b3432c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:29.732270 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e8ca4-a174-4953-88f6-c4c8b3432c0f admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:29.733020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e8ca4-a174-4953-88f6-c4c8b3432c0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:29.735079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e8ca4-a174-4953-88f6-c4c8b3432c0f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:29.755242 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1251/2472] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:45:29 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:45:29.759313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267bb2f-e55a-48c6-a754-587c20e4459d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:29.770093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267bb2f-e55a-48c6-a754-587c20e4459d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:29.775484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34448f4-c2f5-4cea-bfad-7a67836a8c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:29.776182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34448f4-c2f5-4cea-bfad-7a67836a8c02 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:29.776454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34448f4-c2f5-4cea-bfad-7a67836a8c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:29.776736 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34448f4-c2f5-4cea-bfad-7a67836a8c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:29.777470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34448f4-c2f5-4cea-bfad-7a67836a8c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:29.779104 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34448f4-c2f5-4cea-bfad-7a67836a8c02 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:29.790826 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1222/2473] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:45:29.795300 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e8ca4-a174-4953-88f6-c4c8b3432c0f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:29.805341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e8ca4-a174-4953-88f6-c4c8b3432c0f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:29.810233 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcbc7a-c030-477e-93c5-1b012846dc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:29.811146 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcbc7a-c030-477e-93c5-1b012846dc10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:29.811433 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcbc7a-c030-477e-93c5-1b012846dc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:29.811711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcbc7a-c030-477e-93c5-1b012846dc10 admin admin] PATH_INFO: `/v3/projects/4b7329a4d2fa4ce283284326dcae0f88/users/1ad27a9d94324a818a4c98be5967a5d7/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:29.826430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbcbc7a-c030-477e-93c5-1b012846dc10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b7329a4d2fa4ce283284326dcae0f88, user_id=1ad27a9d94324a818a4c98be5967a5d7, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:29.829110 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbcbc7a-c030-477e-93c5-1b012846dc10 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:29.843540 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1252/2474] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:45:29 2022] PUT /identity/v3/projects/4b7329a4d2fa4ce283284326dcae0f88/users/1ad27a9d94324a818a4c98be5967a5d7/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:45:29.847574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34448f4-c2f5-4cea-bfad-7a67836a8c02 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:29.859942 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34448f4-c2f5-4cea-bfad-7a67836a8c02 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:29.916446 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873c667-0707-4146-a071-ba1801a3e99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:29.917035 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873c667-0707-4146-a071-ba1801a3e99a admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:29.917331 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873c667-0707-4146-a071-ba1801a3e99a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:29.917612 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873c667-0707-4146-a071-ba1801a3e99a admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:29.918338 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873c667-0707-4146-a071-ba1801a3e99a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:29.919971 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873c667-0707-4146-a071-ba1801a3e99a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:29.929835 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1223/2475] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:45:29.934371 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcbc7a-c030-477e-93c5-1b012846dc10 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:29.944672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcbc7a-c030-477e-93c5-1b012846dc10 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:29.949315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a3c10-38f2-47b3-af90-e41d23343e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:29.950201 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a3c10-38f2-47b3-af90-e41d23343e90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:29.950483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a3c10-38f2-47b3-af90-e41d23343e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:29.950801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a3c10-38f2-47b3-af90-e41d23343e90 admin admin] PATH_INFO: `/v3/projects/4b7329a4d2fa4ce283284326dcae0f88/users/1ad27a9d94324a818a4c98be5967a5d7/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:29.952750 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a3c10-38f2-47b3-af90-e41d23343e90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b7329a4d2fa4ce283284326dcae0f88, user_id=1ad27a9d94324a818a4c98be5967a5d7, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:29.955398 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a3c10-38f2-47b3-af90-e41d23343e90 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:29.966155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1253/2476] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:45:29 2022] PUT /identity/v3/projects/4b7329a4d2fa4ce283284326dcae0f88/users/1ad27a9d94324a818a4c98be5967a5d7/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:45:29.976889 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea4b93-f07e-49ad-9b60-8493b497493c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:29.977186 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea4b93-f07e-49ad-9b60-8493b497493c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:29.977470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea4b93-f07e-49ad-9b60-8493b497493c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:30.003855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-72ea4b93-f07e-49ad-9b60-8493b497493c None None] MFA Rules not processed for user `1ad27a9d94324a818a4c98be5967a5d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:45:30.005076 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-72ea4b93-f07e-49ad-9b60-8493b497493c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:30.033527 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:30.078928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1224/2477] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:45:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:45:30.091011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a3c10-38f2-47b3-af90-e41d23343e90 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:30.092917 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8b2a3c10-38f2-47b3-af90-e41d23343e90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:30.152255 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a3c10-38f2-47b3-af90-e41d23343e90 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:30.156090 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022acee6-5930-46e5-a4e1-320876de1af7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:30.156743 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022acee6-5930-46e5-a4e1-320876de1af7 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:30.157031 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022acee6-5930-46e5-a4e1-320876de1af7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:30.157358 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022acee6-5930-46e5-a4e1-320876de1af7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:30.166802 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022acee6-5930-46e5-a4e1-320876de1af7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:30.168176 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022acee6-5930-46e5-a4e1-320876de1af7 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:30.177736 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1254/2478] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:45:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:45:30.753855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea4b93-f07e-49ad-9b60-8493b497493c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:30.815727 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea4b93-f07e-49ad-9b60-8493b497493c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:30.820349 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa04ac-859a-4385-8246-caf5a8b12381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:30.821232 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa04ac-859a-4385-8246-caf5a8b12381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:30.821517 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa04ac-859a-4385-8246-caf5a8b12381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:30.821798 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa04ac-859a-4385-8246-caf5a8b12381 admin admin] PATH_INFO: `/v3/users/d114e710cdd44988b07b166d30ef318c` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:30.823088 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa04ac-859a-4385-8246-caf5a8b12381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d114e710cdd44988b07b166d30ef318c)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:30.825019 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa04ac-859a-4385-8246-caf5a8b12381 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:30.872269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f0fa04ac-859a-4385-8246-caf5a8b12381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd114e710cdd44988b07b166d30ef318c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:30.881957 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f0fa04ac-859a-4385-8246-caf5a8b12381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd114e710cdd44988b07b166d30ef318c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:30.890017 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f0fa04ac-859a-4385-8246-caf5a8b12381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd114e710cdd44988b07b166d30ef318c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:30.901916 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f0fa04ac-859a-4385-8246-caf5a8b12381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd114e710cdd44988b07b166d30ef318c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:30.902668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f0fa04ac-859a-4385-8246-caf5a8b12381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd114e710cdd44988b07b166d30ef318c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:30.908548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1225/2479] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:45:30 2022] DELETE /identity/v3/users/d114e710cdd44988b07b166d30ef318c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:45:30.913779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fb7f8-d27a-42c0-bd0b-d5be78f2700a None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:30.914267 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fb7f8-d27a-42c0-bd0b-d5be78f2700a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:30.914379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fb7f8-d27a-42c0-bd0b-d5be78f2700a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:30.933302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6c3fb7f8-d27a-42c0-bd0b-d5be78f2700a None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:45:30.934476 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6c3fb7f8-d27a-42c0-bd0b-d5be78f2700a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:30.993991 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1255/2480] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:45:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:45:31.003719 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa04ac-859a-4385-8246-caf5a8b12381 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:31.005149 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f0fa04ac-859a-4385-8246-caf5a8b12381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:31.051696 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:31.065400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa04ac-859a-4385-8246-caf5a8b12381 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:31.069090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367f8c8-cd13-41c0-a9e4-ebb415dfa96b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:31.069802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367f8c8-cd13-41c0-a9e4-ebb415dfa96b service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:31.070085 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367f8c8-cd13-41c0-a9e4-ebb415dfa96b service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:31.070359 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367f8c8-cd13-41c0-a9e4-ebb415dfa96b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:31.079657 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367f8c8-cd13-41c0-a9e4-ebb415dfa96b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:31.081091 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367f8c8-cd13-41c0-a9e4-ebb415dfa96b service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:31.090701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1226/2481] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:45:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:45:31.363082 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fb7f8-d27a-42c0-bd0b-d5be78f2700a None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:31.364500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6c3fb7f8-d27a-42c0-bd0b-d5be78f2700a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:31.377795 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fb7f8-d27a-42c0-bd0b-d5be78f2700a None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:31.383726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b969944-10c4-4588-aec0-eb6001543802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:31.385153 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b969944-10c4-4588-aec0-eb6001543802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:31.385153 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b969944-10c4-4588-aec0-eb6001543802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:31.385407 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b969944-10c4-4588-aec0-eb6001543802 admin admin] PATH_INFO: `/v3/projects/53a056f221f84fd3a165dcb180033de6` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:31.386623 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b969944-10c4-4588-aec0-eb6001543802 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53a056f221f84fd3a165dcb180033de6)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:31.389208 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b969944-10c4-4588-aec0-eb6001543802 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:31.403857 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:31.444630 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8b969944-10c4-4588-aec0-eb6001543802 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53a056f221f84fd3a165dcb180033de6', 'request_id': 'req-8b969944-10c4-4588-aec0-eb6001543802'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:31.450585 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8b969944-10c4-4588-aec0-eb6001543802 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53a056f221f84fd3a165dcb180033de6', 'request_id': 'req-8b969944-10c4-4588-aec0-eb6001543802'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:31.456853 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8b969944-10c4-4588-aec0-eb6001543802 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53a056f221f84fd3a165dcb180033de6', 'request_id': 'req-8b969944-10c4-4588-aec0-eb6001543802'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:31.469366 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8b969944-10c4-4588-aec0-eb6001543802 admin admin] The token cache is being invalidate because project 53a056f221f84fd3a165dcb180033de6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:45:31.469702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8b969944-10c4-4588-aec0-eb6001543802 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:31.470348 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-8b969944-10c4-4588-aec0-eb6001543802 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:31.471912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1256/2482] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:45:31 2022] DELETE /identity/v3/projects/53a056f221f84fd3a165dcb180033de6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:45:31.476529 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367f8c8-cd13-41c0-a9e4-ebb415dfa96b service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:31.486504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367f8c8-cd13-41c0-a9e4-ebb415dfa96b service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:31.546325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b757725-72c3-4efa-9106-43fb7665bf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:31.547438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b757725-72c3-4efa-9106-43fb7665bf14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:31.547721 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b757725-72c3-4efa-9106-43fb7665bf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:31.547997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b757725-72c3-4efa-9106-43fb7665bf14 admin admin] PATH_INFO: `/v3/users/1ad27a9d94324a818a4c98be5967a5d7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:31.549252 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b757725-72c3-4efa-9106-43fb7665bf14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ad27a9d94324a818a4c98be5967a5d7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:31.551234 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b757725-72c3-4efa-9106-43fb7665bf14 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:31.604806 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3b757725-72c3-4efa-9106-43fb7665bf14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ad27a9d94324a818a4c98be5967a5d7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:31.613582 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3b757725-72c3-4efa-9106-43fb7665bf14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ad27a9d94324a818a4c98be5967a5d7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:31.623774 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3b757725-72c3-4efa-9106-43fb7665bf14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ad27a9d94324a818a4c98be5967a5d7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:31.638622 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3b757725-72c3-4efa-9106-43fb7665bf14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ad27a9d94324a818a4c98be5967a5d7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:31.639361 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3b757725-72c3-4efa-9106-43fb7665bf14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ad27a9d94324a818a4c98be5967a5d7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:31.645810 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1227/2483] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:45:31 2022] DELETE /identity/v3/users/1ad27a9d94324a818a4c98be5967a5d7 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:45:31.922874 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b969944-10c4-4588-aec0-eb6001543802 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:31.925741 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8b969944-10c4-4588-aec0-eb6001543802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:31.941459 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b969944-10c4-4588-aec0-eb6001543802 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:32.001586 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ef4cb-9240-409f-8c0d-4be300f27970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:32.003022 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ef4cb-9240-409f-8c0d-4be300f27970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:32.003301 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ef4cb-9240-409f-8c0d-4be300f27970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:32.003583 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ef4cb-9240-409f-8c0d-4be300f27970 admin admin] PATH_INFO: `/v3/projects/4b7329a4d2fa4ce283284326dcae0f88` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:32.004818 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ef4cb-9240-409f-8c0d-4be300f27970 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b7329a4d2fa4ce283284326dcae0f88)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:32.007380 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ef4cb-9240-409f-8c0d-4be300f27970 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:32.055180 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6f4ef4cb-9240-409f-8c0d-4be300f27970 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b7329a4d2fa4ce283284326dcae0f88', 'request_id': 'req-6f4ef4cb-9240-409f-8c0d-4be300f27970'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:32.060900 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6f4ef4cb-9240-409f-8c0d-4be300f27970 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b7329a4d2fa4ce283284326dcae0f88', 'request_id': 'req-6f4ef4cb-9240-409f-8c0d-4be300f27970'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:32.067021 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6f4ef4cb-9240-409f-8c0d-4be300f27970 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b7329a4d2fa4ce283284326dcae0f88', 'request_id': 'req-6f4ef4cb-9240-409f-8c0d-4be300f27970'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:32.080691 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6f4ef4cb-9240-409f-8c0d-4be300f27970 admin admin] The token cache is being invalidate because project 4b7329a4d2fa4ce283284326dcae0f88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:45:32.080988 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6f4ef4cb-9240-409f-8c0d-4be300f27970 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:32.081628 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6f4ef4cb-9240-409f-8c0d-4be300f27970 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:32.083337 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1257/2484] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:45:31 2022] DELETE /identity/v3/projects/4b7329a4d2fa4ce283284326dcae0f88 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:45:32.088466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8bf29-fa3f-4c50-bf72-4eb56a339f49 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:32.088967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8bf29-fa3f-4c50-bf72-4eb56a339f49 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:32.089089 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8bf29-fa3f-4c50-bf72-4eb56a339f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:32.107029 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-cfd8bf29-fa3f-4c50-bf72-4eb56a339f49 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:45:32.108266 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cfd8bf29-fa3f-4c50-bf72-4eb56a339f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:32.173324 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1228/2485] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:45:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:45:32.185251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf8eda-78f5-43f1-940a-a880df1223be None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:32.185725 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf8eda-78f5-43f1-940a-a880df1223be None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:32.186189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf8eda-78f5-43f1-940a-a880df1223be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:32.212924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-1cdf8eda-78f5-43f1-940a-a880df1223be None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:45:32.214761 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1cdf8eda-78f5-43f1-940a-a880df1223be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:32.224393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1258/2486] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:45:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:45:32.229505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8bf29-fa3f-4c50-bf72-4eb56a339f49 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:32.235619 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:32.239771 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8bf29-fa3f-4c50-bf72-4eb56a339f49 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:32.246197 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65512c-a4fc-4689-a4e1-34bc295cb9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:32.246921 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65512c-a4fc-4689-a4e1-34bc295cb9fd admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:32.247385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65512c-a4fc-4689-a4e1-34bc295cb9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:32.247831 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65512c-a4fc-4689-a4e1-34bc295cb9fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:32.248736 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65512c-a4fc-4689-a4e1-34bc295cb9fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:45:32.249046 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65512c-a4fc-4689-a4e1-34bc295cb9fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:32.250633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65512c-a4fc-4689-a4e1-34bc295cb9fd admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:32.263072 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1229/2487] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:45:32.267212 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf8eda-78f5-43f1-940a-a880df1223be None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:32.276473 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf8eda-78f5-43f1-940a-a880df1223be None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:32.281141 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8dbb0-9a08-43d8-bbe6-e9663d65c40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:32.281794 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8dbb0-9a08-43d8-bbe6-e9663d65c40f admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:32.282080 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8dbb0-9a08-43d8-bbe6-e9663d65c40f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:32.282358 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8dbb0-9a08-43d8-bbe6-e9663d65c40f admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:32.283111 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8dbb0-9a08-43d8-bbe6-e9663d65c40f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:32.285000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8dbb0-9a08-43d8-bbe6-e9663d65c40f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:32.311226 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1259/2488] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:45:32 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:45:32.315310 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65512c-a4fc-4689-a4e1-34bc295cb9fd admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:32.325164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65512c-a4fc-4689-a4e1-34bc295cb9fd admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:32.380378 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2e70b-224d-422d-b744-fe06bb1eaedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:32.381049 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2e70b-224d-422d-b744-fe06bb1eaedb admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:32.381337 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2e70b-224d-422d-b744-fe06bb1eaedb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:32.381612 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2e70b-224d-422d-b744-fe06bb1eaedb admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:32.382971 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2e70b-224d-422d-b744-fe06bb1eaedb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:32.384880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2e70b-224d-422d-b744-fe06bb1eaedb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:32.405835 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1230/2489] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:45:32 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:45:32.409358 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8dbb0-9a08-43d8-bbe6-e9663d65c40f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:32.414882 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:32.421491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8dbb0-9a08-43d8-bbe6-e9663d65c40f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:32.427286 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2134aae-ccfc-4aae-a946-e6801dbb2f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:32.427995 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2134aae-ccfc-4aae-a946-e6801dbb2f89 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:32.428441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2134aae-ccfc-4aae-a946-e6801dbb2f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:32.428766 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2134aae-ccfc-4aae-a946-e6801dbb2f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:32.429504 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2134aae-ccfc-4aae-a946-e6801dbb2f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:32.431498 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2134aae-ccfc-4aae-a946-e6801dbb2f89 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:32.442080 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1260/2490] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:45:32.446634 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2e70b-224d-422d-b744-fe06bb1eaedb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:32.456553 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2e70b-224d-422d-b744-fe06bb1eaedb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:32.461922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc6214-0671-43ab-905f-58892f04cc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:32.462584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc6214-0671-43ab-905f-58892f04cc81 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:32.462905 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc6214-0671-43ab-905f-58892f04cc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:32.463189 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc6214-0671-43ab-905f-58892f04cc81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:32.463916 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc6214-0671-43ab-905f-58892f04cc81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:32.465509 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc6214-0671-43ab-905f-58892f04cc81 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:32.475482 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1231/2491] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:45:32.479805 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2134aae-ccfc-4aae-a946-e6801dbb2f89 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:32.489630 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2134aae-ccfc-4aae-a946-e6801dbb2f89 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:32.494789 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde988c-69c6-47ad-a2ca-dc661a289e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:32.495741 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde988c-69c6-47ad-a2ca-dc661a289e99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:32.496022 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde988c-69c6-47ad-a2ca-dc661a289e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:32.496296 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde988c-69c6-47ad-a2ca-dc661a289e99 admin admin] PATH_INFO: `/v3/projects/543ee4e69ef84b6db1e871e4fcae7e90/users/f92d96deea804105984afd262f33574a/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:32.514222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde988c-69c6-47ad-a2ca-dc661a289e99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=543ee4e69ef84b6db1e871e4fcae7e90, user_id=f92d96deea804105984afd262f33574a, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:32.516881 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde988c-69c6-47ad-a2ca-dc661a289e99 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:32.535525 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1261/2492] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:45:32 2022] PUT /identity/v3/projects/543ee4e69ef84b6db1e871e4fcae7e90/users/f92d96deea804105984afd262f33574a/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:45:32.546429 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f0893-d583-406e-9429-68033d2bd0ec None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:32.546754 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f0893-d583-406e-9429-68033d2bd0ec None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:32.547041 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f0893-d583-406e-9429-68033d2bd0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:32.583071 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8b4f0893-d583-406e-9429-68033d2bd0ec None None] MFA Rules not processed for user `f92d96deea804105984afd262f33574a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:45:32.584341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8b4f0893-d583-406e-9429-68033d2bd0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:32.651134 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1232/2493] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:45:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:45:32.655634 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde988c-69c6-47ad-a2ca-dc661a289e99 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:32.716477 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde988c-69c6-47ad-a2ca-dc661a289e99 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:32.721067 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e993a1a-b8b1-4a83-bd2d-b601fb2a9028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:32.721729 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e993a1a-b8b1-4a83-bd2d-b601fb2a9028 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:32.722015 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e993a1a-b8b1-4a83-bd2d-b601fb2a9028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:32.722296 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e993a1a-b8b1-4a83-bd2d-b601fb2a9028 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:32.723030 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e993a1a-b8b1-4a83-bd2d-b601fb2a9028 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:32.724945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e993a1a-b8b1-4a83-bd2d-b601fb2a9028 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:32.750426 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1262/2494] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:45:32 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 14:45:32.754540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f0893-d583-406e-9429-68033d2bd0ec None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:32.766216 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f0893-d583-406e-9429-68033d2bd0ec None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:32.824481 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e86ff-e396-415c-84e4-6404e9bff6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:32.825175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e86ff-e396-415c-84e4-6404e9bff6c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:32.825461 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e86ff-e396-415c-84e4-6404e9bff6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:32.825740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e86ff-e396-415c-84e4-6404e9bff6c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:32.826442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e86ff-e396-415c-84e4-6404e9bff6c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:32.828418 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e86ff-e396-415c-84e4-6404e9bff6c4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:32.835954 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:32.850611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1233/2495] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:45:32 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:45:32.867395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e993a1a-b8b1-4a83-bd2d-b601fb2a9028 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:32.878904 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e993a1a-b8b1-4a83-bd2d-b601fb2a9028 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:32.884297 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171d58a-1a92-47d1-a36f-08a3fa464a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:32.884979 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171d58a-1a92-47d1-a36f-08a3fa464a80 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:32.885266 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171d58a-1a92-47d1-a36f-08a3fa464a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:32.885546 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171d58a-1a92-47d1-a36f-08a3fa464a80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:32.886295 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171d58a-1a92-47d1-a36f-08a3fa464a80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:32.887942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171d58a-1a92-47d1-a36f-08a3fa464a80 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:32.899427 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1263/2496] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:45:32.903396 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e86ff-e396-415c-84e4-6404e9bff6c4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:32.913595 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e86ff-e396-415c-84e4-6404e9bff6c4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:32.918286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ffeef-69de-4937-acfc-fb0c0566aef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:32.918960 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ffeef-69de-4937-acfc-fb0c0566aef1 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:32.919252 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ffeef-69de-4937-acfc-fb0c0566aef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:32.919532 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ffeef-69de-4937-acfc-fb0c0566aef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:32.920264 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ffeef-69de-4937-acfc-fb0c0566aef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:32.921985 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ffeef-69de-4937-acfc-fb0c0566aef1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:32.936932 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1234/2497] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:45:32.941144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171d58a-1a92-47d1-a36f-08a3fa464a80 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:32.951827 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171d58a-1a92-47d1-a36f-08a3fa464a80 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:32.956518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf5ef3-193e-4104-ad19-e577ba25eabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:32.957396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf5ef3-193e-4104-ad19-e577ba25eabf admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:32.957678 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf5ef3-193e-4104-ad19-e577ba25eabf admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:32.957954 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf5ef3-193e-4104-ad19-e577ba25eabf admin admin] PATH_INFO: `/v3/projects/79f40c2b62e9431d89d5aa3acaa4e807/users/5862620b554e43aeab50ea003d705a11/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:32.972658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf5ef3-193e-4104-ad19-e577ba25eabf admin admin] RBAC: Authorizing `identity:create_grant(project_id=79f40c2b62e9431d89d5aa3acaa4e807, user_id=5862620b554e43aeab50ea003d705a11, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:32.975381 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf5ef3-193e-4104-ad19-e577ba25eabf admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:32.989119 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1264/2498] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:45:32 2022] PUT /identity/v3/projects/79f40c2b62e9431d89d5aa3acaa4e807/users/5862620b554e43aeab50ea003d705a11/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:45:33.000004 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c290e-055f-4590-9e4a-c6df28fc1002 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:33.000302 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c290e-055f-4590-9e4a-c6df28fc1002 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:33.000587 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c290e-055f-4590-9e4a-c6df28fc1002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:33.027768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-839c290e-055f-4590-9e4a-c6df28fc1002 None None] MFA Rules not processed for user `5862620b554e43aeab50ea003d705a11`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:45:33.028954 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-839c290e-055f-4590-9e4a-c6df28fc1002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:33.094014 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1235/2499] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:45:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:45:33.098528 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf5ef3-193e-4104-ad19-e577ba25eabf admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:33.161403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf5ef3-193e-4104-ad19-e577ba25eabf admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:33.166563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7895ae-eaab-4106-9800-3b26f8063d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:33.167245 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7895ae-eaab-4106-9800-3b26f8063d90 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:33.167524 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7895ae-eaab-4106-9800-3b26f8063d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:33.167799 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7895ae-eaab-4106-9800-3b26f8063d90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:33.168491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7895ae-eaab-4106-9800-3b26f8063d90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:33.170390 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7895ae-eaab-4106-9800-3b26f8063d90 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:33.195974 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1265/2500] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:45:33 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:45:33.200078 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c290e-055f-4590-9e4a-c6df28fc1002 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:33.211068 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c290e-055f-4590-9e4a-c6df28fc1002 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:33.270596 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566693be-7073-43bb-81ff-6149f2a203af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:33.271302 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566693be-7073-43bb-81ff-6149f2a203af admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:33.271583 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566693be-7073-43bb-81ff-6149f2a203af admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:33.271854 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566693be-7073-43bb-81ff-6149f2a203af admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:33.272580 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566693be-7073-43bb-81ff-6149f2a203af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:33.274486 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566693be-7073-43bb-81ff-6149f2a203af admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:33.295799 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1236/2501] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:45:33 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:45:33.299674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7895ae-eaab-4106-9800-3b26f8063d90 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:33.305645 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:33.312283 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7895ae-eaab-4106-9800-3b26f8063d90 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:33.317791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578766bc-0d17-4fca-abe2-e9aaa911638f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:33.318447 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578766bc-0d17-4fca-abe2-e9aaa911638f admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:33.318771 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578766bc-0d17-4fca-abe2-e9aaa911638f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:33.319053 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578766bc-0d17-4fca-abe2-e9aaa911638f admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:33.319788 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578766bc-0d17-4fca-abe2-e9aaa911638f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:33.321409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578766bc-0d17-4fca-abe2-e9aaa911638f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:33.332270 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1266/2502] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 14:45:33.336593 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566693be-7073-43bb-81ff-6149f2a203af admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:33.346894 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566693be-7073-43bb-81ff-6149f2a203af admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:33.352023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab841f1-ede3-4de1-9e74-4ef35862779c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:33.352919 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab841f1-ede3-4de1-9e74-4ef35862779c admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:33.353204 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab841f1-ede3-4de1-9e74-4ef35862779c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:33.353497 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab841f1-ede3-4de1-9e74-4ef35862779c admin admin] PATH_INFO: `/v3/projects/1e29385fb6d74d6d8e37c1e755e597da/users/b86743a2fda348dcae59fed5d9c5f677/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:33.368793 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab841f1-ede3-4de1-9e74-4ef35862779c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e29385fb6d74d6d8e37c1e755e597da, user_id=b86743a2fda348dcae59fed5d9c5f677, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:33.371465 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab841f1-ede3-4de1-9e74-4ef35862779c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:33.385636 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1237/2503] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:45:33 2022] PUT /identity/v3/projects/1e29385fb6d74d6d8e37c1e755e597da/users/b86743a2fda348dcae59fed5d9c5f677/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:45:33.389654 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578766bc-0d17-4fca-abe2-e9aaa911638f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:33.401386 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578766bc-0d17-4fca-abe2-e9aaa911638f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:33.462217 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ead10-29e0-484c-b4d6-d3421da88f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:33.462947 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ead10-29e0-484c-b4d6-d3421da88f0f admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:33.463235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ead10-29e0-484c-b4d6-d3421da88f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:33.463518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ead10-29e0-484c-b4d6-d3421da88f0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:33.464251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ead10-29e0-484c-b4d6-d3421da88f0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:33.465870 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ead10-29e0-484c-b4d6-d3421da88f0f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:33.476823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1267/2504] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:45:33.481005 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab841f1-ede3-4de1-9e74-4ef35862779c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:33.491791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab841f1-ede3-4de1-9e74-4ef35862779c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:33.496632 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f8611-a072-4079-bbe9-3a510f0b7705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:33.497539 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f8611-a072-4079-bbe9-3a510f0b7705 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:33.497822 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f8611-a072-4079-bbe9-3a510f0b7705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:33.498107 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f8611-a072-4079-bbe9-3a510f0b7705 admin admin] PATH_INFO: `/v3/projects/1e29385fb6d74d6d8e37c1e755e597da/users/b86743a2fda348dcae59fed5d9c5f677/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:33.500134 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3f8611-a072-4079-bbe9-3a510f0b7705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e29385fb6d74d6d8e37c1e755e597da, user_id=b86743a2fda348dcae59fed5d9c5f677, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:33.502769 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3f8611-a072-4079-bbe9-3a510f0b7705 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:33.513729 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1238/2505] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:45:33 2022] PUT /identity/v3/projects/1e29385fb6d74d6d8e37c1e755e597da/users/b86743a2fda348dcae59fed5d9c5f677/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:45:33.525592 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26688bcb-7768-47f3-ad84-bceb07814af4 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:33.525882 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26688bcb-7768-47f3-ad84-bceb07814af4 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:33.526164 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26688bcb-7768-47f3-ad84-bceb07814af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:33.554035 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-26688bcb-7768-47f3-ad84-bceb07814af4 None None] MFA Rules not processed for user `b86743a2fda348dcae59fed5d9c5f677`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:45:33.555415 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-26688bcb-7768-47f3-ad84-bceb07814af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:33.641760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1268/2506] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:45:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:45:33.653614 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f8611-a072-4079-bbe9-3a510f0b7705 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:33.655377 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fb3f8611-a072-4079-bbe9-3a510f0b7705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:33.678763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:33.717699 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f8611-a072-4079-bbe9-3a510f0b7705 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:33.721524 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc257df-276d-417d-b014-2f665c91f1ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:33.722185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc257df-276d-417d-b014-2f665c91f1ae service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:33.722472 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc257df-276d-417d-b014-2f665c91f1ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:33.722781 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc257df-276d-417d-b014-2f665c91f1ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:33.781833 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc257df-276d-417d-b014-2f665c91f1ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:33.783327 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc257df-276d-417d-b014-2f665c91f1ae service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:33.793779 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1239/2507] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:45:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:45:35.322149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26688bcb-7768-47f3-ad84-bceb07814af4 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:35.323574 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-26688bcb-7768-47f3-ad84-bceb07814af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:35.336128 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26688bcb-7768-47f3-ad84-bceb07814af4 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:35.339686 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91c8b9-8fc2-49fe-b512-b9135279c7e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:35.340404 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91c8b9-8fc2-49fe-b512-b9135279c7e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:35.340688 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91c8b9-8fc2-49fe-b512-b9135279c7e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:35.340967 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91c8b9-8fc2-49fe-b512-b9135279c7e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:35.396738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91c8b9-8fc2-49fe-b512-b9135279c7e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:35.398723 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91c8b9-8fc2-49fe-b512-b9135279c7e3 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:35.407796 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1269/2508] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:45:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:45:35.940251 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc257df-276d-417d-b014-2f665c91f1ae service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:35.948994 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc257df-276d-417d-b014-2f665c91f1ae service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:35.952674 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccf2875-c5ed-44f3-908f-76145ef615ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:35.953336 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf2875-c5ed-44f3-908f-76145ef615ea service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:35.953623 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf2875-c5ed-44f3-908f-76145ef615ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:35.953905 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf2875-c5ed-44f3-908f-76145ef615ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:35.964394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccf2875-c5ed-44f3-908f-76145ef615ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:35.965972 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccf2875-c5ed-44f3-908f-76145ef615ea service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:35.977085 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1240/2509] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:45:35 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:45:39.878750 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91c8b9-8fc2-49fe-b512-b9135279c7e3 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:39.891571 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=100 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:39.942222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91c8b9-8fc2-49fe-b512-b9135279c7e3 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:39.947360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14811e7-aa3b-472c-af04-cec65102056b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:39.948240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14811e7-aa3b-472c-af04-cec65102056b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:39.948521 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14811e7-aa3b-472c-af04-cec65102056b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:39.948795 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14811e7-aa3b-472c-af04-cec65102056b admin admin] PATH_INFO: `/v3/users/f92d96deea804105984afd262f33574a` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:39.949976 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14811e7-aa3b-472c-af04-cec65102056b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f92d96deea804105984afd262f33574a)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:39.951929 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14811e7-aa3b-472c-af04-cec65102056b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:40.001658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d14811e7-aa3b-472c-af04-cec65102056b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f92d96deea804105984afd262f33574a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:40.009519 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d14811e7-aa3b-472c-af04-cec65102056b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f92d96deea804105984afd262f33574a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:40.010302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d14811e7-aa3b-472c-af04-cec65102056b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f92d96deea804105984afd262f33574a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:40.023717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d14811e7-aa3b-472c-af04-cec65102056b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f92d96deea804105984afd262f33574a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:40.024402 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-d14811e7-aa3b-472c-af04-cec65102056b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f92d96deea804105984afd262f33574a'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:40.036644 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1270/2510] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:45:39 2022] DELETE /identity/v3/users/f92d96deea804105984afd262f33574a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:45:40.041139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea71b2c-9122-4570-8f83-2c1021b0ed98 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:40.041634 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea71b2c-9122-4570-8f83-2c1021b0ed98 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:40.041746 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea71b2c-9122-4570-8f83-2c1021b0ed98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:40.059108 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0ea71b2c-9122-4570-8f83-2c1021b0ed98 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:45:40.060406 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0ea71b2c-9122-4570-8f83-2c1021b0ed98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:40.120598 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1241/2511] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:45:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:45:40.128424 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14811e7-aa3b-472c-af04-cec65102056b admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:40.130097 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d14811e7-aa3b-472c-af04-cec65102056b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:40.187869 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14811e7-aa3b-472c-af04-cec65102056b admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:40.191488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be8f0f-1ac4-4c31-a033-d7be0ffadd46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:40.192135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be8f0f-1ac4-4c31-a033-d7be0ffadd46 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:40.192416 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be8f0f-1ac4-4c31-a033-d7be0ffadd46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:40.192692 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be8f0f-1ac4-4c31-a033-d7be0ffadd46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:40.202235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be8f0f-1ac4-4c31-a033-d7be0ffadd46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:40.203693 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be8f0f-1ac4-4c31-a033-d7be0ffadd46 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:40.212934 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1271/2512] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:45:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:45:40.492181 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea71b2c-9122-4570-8f83-2c1021b0ed98 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:40.493696 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0ea71b2c-9122-4570-8f83-2c1021b0ed98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:40.508542 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea71b2c-9122-4570-8f83-2c1021b0ed98 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:40.513582 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682c4bb-d0c4-47cc-bfec-e41aeff36f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:40.514627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682c4bb-d0c4-47cc-bfec-e41aeff36f1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:40.514948 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682c4bb-d0c4-47cc-bfec-e41aeff36f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:40.515230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682c4bb-d0c4-47cc-bfec-e41aeff36f1d admin admin] PATH_INFO: `/v3/projects/543ee4e69ef84b6db1e871e4fcae7e90` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:40.516576 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2682c4bb-d0c4-47cc-bfec-e41aeff36f1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=543ee4e69ef84b6db1e871e4fcae7e90)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:40.518408 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2682c4bb-d0c4-47cc-bfec-e41aeff36f1d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:40.565369 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2682c4bb-d0c4-47cc-bfec-e41aeff36f1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '543ee4e69ef84b6db1e871e4fcae7e90', 'request_id': 'req-2682c4bb-d0c4-47cc-bfec-e41aeff36f1d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:40.569943 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:40.571398 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2682c4bb-d0c4-47cc-bfec-e41aeff36f1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '543ee4e69ef84b6db1e871e4fcae7e90', 'request_id': 'req-2682c4bb-d0c4-47cc-bfec-e41aeff36f1d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:40.579072 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2682c4bb-d0c4-47cc-bfec-e41aeff36f1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '543ee4e69ef84b6db1e871e4fcae7e90', 'request_id': 'req-2682c4bb-d0c4-47cc-bfec-e41aeff36f1d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:40.591817 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2682c4bb-d0c4-47cc-bfec-e41aeff36f1d admin admin] The token cache is being invalidate because project 543ee4e69ef84b6db1e871e4fcae7e90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:45:40.592109 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2682c4bb-d0c4-47cc-bfec-e41aeff36f1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:40.592744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2682c4bb-d0c4-47cc-bfec-e41aeff36f1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:40.594308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1242/2513] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:45:40 2022] DELETE /identity/v3/projects/543ee4e69ef84b6db1e871e4fcae7e90 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:45:40.598414 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be8f0f-1ac4-4c31-a033-d7be0ffadd46 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:40.613961 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be8f0f-1ac4-4c31-a033-d7be0ffadd46 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:40.645839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:40.680228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e04f4be-24ea-43f7-a931-8960e160ba79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:40.681124 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04f4be-24ea-43f7-a931-8960e160ba79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:40.681408 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04f4be-24ea-43f7-a931-8960e160ba79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:40.681686 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04f4be-24ea-43f7-a931-8960e160ba79 admin admin] PATH_INFO: `/v3/users/5862620b554e43aeab50ea003d705a11` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:40.683190 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e04f4be-24ea-43f7-a931-8960e160ba79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5862620b554e43aeab50ea003d705a11)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:40.685161 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e04f4be-24ea-43f7-a931-8960e160ba79 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:40.730912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0e04f4be-24ea-43f7-a931-8960e160ba79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5862620b554e43aeab50ea003d705a11'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:40.738214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0e04f4be-24ea-43f7-a931-8960e160ba79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5862620b554e43aeab50ea003d705a11'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:40.738848 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0e04f4be-24ea-43f7-a931-8960e160ba79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5862620b554e43aeab50ea003d705a11'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:40.751425 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0e04f4be-24ea-43f7-a931-8960e160ba79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5862620b554e43aeab50ea003d705a11'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:40.752081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0e04f4be-24ea-43f7-a931-8960e160ba79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5862620b554e43aeab50ea003d705a11'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:40.764404 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1272/2514] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:45:40 2022] DELETE /identity/v3/users/5862620b554e43aeab50ea003d705a11 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 14:45:41.040740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682c4bb-d0c4-47cc-bfec-e41aeff36f1d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:41.042315 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2682c4bb-d0c4-47cc-bfec-e41aeff36f1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:41.056075 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682c4bb-d0c4-47cc-bfec-e41aeff36f1d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:41.115893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da430c06-b7ce-412d-b985-2d097df665e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:41.116771 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da430c06-b7ce-412d-b985-2d097df665e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:41.117051 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da430c06-b7ce-412d-b985-2d097df665e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:41.117322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da430c06-b7ce-412d-b985-2d097df665e8 admin admin] PATH_INFO: `/v3/projects/79f40c2b62e9431d89d5aa3acaa4e807` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:41.118520 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da430c06-b7ce-412d-b985-2d097df665e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79f40c2b62e9431d89d5aa3acaa4e807)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:41.120446 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da430c06-b7ce-412d-b985-2d097df665e8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:41.168749 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-da430c06-b7ce-412d-b985-2d097df665e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79f40c2b62e9431d89d5aa3acaa4e807', 'request_id': 'req-da430c06-b7ce-412d-b985-2d097df665e8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:41.174090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-da430c06-b7ce-412d-b985-2d097df665e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79f40c2b62e9431d89d5aa3acaa4e807', 'request_id': 'req-da430c06-b7ce-412d-b985-2d097df665e8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:41.180356 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-da430c06-b7ce-412d-b985-2d097df665e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79f40c2b62e9431d89d5aa3acaa4e807', 'request_id': 'req-da430c06-b7ce-412d-b985-2d097df665e8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:41.193011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-da430c06-b7ce-412d-b985-2d097df665e8 admin admin] The token cache is being invalidate because project 79f40c2b62e9431d89d5aa3acaa4e807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:45:41.193302 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-da430c06-b7ce-412d-b985-2d097df665e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:41.193981 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-da430c06-b7ce-412d-b985-2d097df665e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:41.195642 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1243/2515] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:45:41 2022] DELETE /identity/v3/projects/79f40c2b62e9431d89d5aa3acaa4e807 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:45:41.199778 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e04f4be-24ea-43f7-a931-8960e160ba79 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:41.201202 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0e04f4be-24ea-43f7-a931-8960e160ba79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:41.214505 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e04f4be-24ea-43f7-a931-8960e160ba79 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:41.269736 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc721d6e-7f6c-4c73-aefa-a9e692e01262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:41.270619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc721d6e-7f6c-4c73-aefa-a9e692e01262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:41.270936 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc721d6e-7f6c-4c73-aefa-a9e692e01262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:41.271221 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc721d6e-7f6c-4c73-aefa-a9e692e01262 admin admin] PATH_INFO: `/v3/users/b86743a2fda348dcae59fed5d9c5f677` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:41.273119 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc721d6e-7f6c-4c73-aefa-a9e692e01262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b86743a2fda348dcae59fed5d9c5f677)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:41.275580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc721d6e-7f6c-4c73-aefa-a9e692e01262 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:41.324125 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-dc721d6e-7f6c-4c73-aefa-a9e692e01262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b86743a2fda348dcae59fed5d9c5f677'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:41.330902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-dc721d6e-7f6c-4c73-aefa-a9e692e01262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b86743a2fda348dcae59fed5d9c5f677'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:41.331529 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-dc721d6e-7f6c-4c73-aefa-a9e692e01262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b86743a2fda348dcae59fed5d9c5f677'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:41.332684 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:41.348085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-dc721d6e-7f6c-4c73-aefa-a9e692e01262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b86743a2fda348dcae59fed5d9c5f677'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:41.348772 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-dc721d6e-7f6c-4c73-aefa-a9e692e01262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b86743a2fda348dcae59fed5d9c5f677'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:41.361065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1273/2516] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:45:41 2022] DELETE /identity/v3/users/b86743a2fda348dcae59fed5d9c5f677 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:45:41.631137 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da430c06-b7ce-412d-b985-2d097df665e8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:41.633097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-da430c06-b7ce-412d-b985-2d097df665e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:41.645742 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da430c06-b7ce-412d-b985-2d097df665e8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:41.708780 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59954a-3273-469a-a4af-90c85195949d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:41.709587 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59954a-3273-469a-a4af-90c85195949d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:41.709870 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59954a-3273-469a-a4af-90c85195949d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:41.710147 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59954a-3273-469a-a4af-90c85195949d admin admin] PATH_INFO: `/v3/projects/1e29385fb6d74d6d8e37c1e755e597da` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:41.711343 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59954a-3273-469a-a4af-90c85195949d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e29385fb6d74d6d8e37c1e755e597da)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:41.713262 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59954a-3273-469a-a4af-90c85195949d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:41.729805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:41.761162 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6d59954a-3273-469a-a4af-90c85195949d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e29385fb6d74d6d8e37c1e755e597da', 'request_id': 'req-6d59954a-3273-469a-a4af-90c85195949d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:41.766624 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6d59954a-3273-469a-a4af-90c85195949d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e29385fb6d74d6d8e37c1e755e597da', 'request_id': 'req-6d59954a-3273-469a-a4af-90c85195949d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:41.772830 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6d59954a-3273-469a-a4af-90c85195949d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e29385fb6d74d6d8e37c1e755e597da', 'request_id': 'req-6d59954a-3273-469a-a4af-90c85195949d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:41.784891 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6d59954a-3273-469a-a4af-90c85195949d admin admin] The token cache is being invalidate because project 1e29385fb6d74d6d8e37c1e755e597da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:45:41.785180 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6d59954a-3273-469a-a4af-90c85195949d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:41.785799 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6d59954a-3273-469a-a4af-90c85195949d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:45:41.787371 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1244/2517] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:45:41 2022] DELETE /identity/v3/projects/1e29385fb6d74d6d8e37c1e755e597da => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:45:41.792395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7231f8-5ccb-4a8d-8ceb-83d6801fcabd None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:41.792684 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7231f8-5ccb-4a8d-8ceb-83d6801fcabd None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:41.792960 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7231f8-5ccb-4a8d-8ceb-83d6801fcabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:41.811760 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-2c7231f8-5ccb-4a8d-8ceb-83d6801fcabd None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:45:41.812990 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2c7231f8-5ccb-4a8d-8ceb-83d6801fcabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:41.870893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1274/2518] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:45:41.878885 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293ffb1-ff42-4e94-81a2-7fd6c72ac333 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:41.879181 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293ffb1-ff42-4e94-81a2-7fd6c72ac333 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:41.879466 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293ffb1-ff42-4e94-81a2-7fd6c72ac333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:41.898317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c293ffb1-ff42-4e94-81a2-7fd6c72ac333 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:45:41.899576 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c293ffb1-ff42-4e94-81a2-7fd6c72ac333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:41.907494 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1245/2519] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:45:41.911635 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7231f8-5ccb-4a8d-8ceb-83d6801fcabd None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:41.921291 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7231f8-5ccb-4a8d-8ceb-83d6801fcabd None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:41.926095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ba977-66b3-4286-b957-393cbac07630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:41.926777 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ba977-66b3-4286-b957-393cbac07630 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:41.927062 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ba977-66b3-4286-b957-393cbac07630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:41.927333 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ba977-66b3-4286-b957-393cbac07630 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:41.928057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ba977-66b3-4286-b957-393cbac07630 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:45:41.928884 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ba977-66b3-4286-b957-393cbac07630 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:41.930506 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ba977-66b3-4286-b957-393cbac07630 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:41.942231 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1275/2520] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:45:41.947469 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293ffb1-ff42-4e94-81a2-7fd6c72ac333 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:41.956746 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293ffb1-ff42-4e94-81a2-7fd6c72ac333 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:41.961569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b839f6-45f0-44d8-b1e5-f7fc6b54fa34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:41.962170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b839f6-45f0-44d8-b1e5-f7fc6b54fa34 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:41.962487 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b839f6-45f0-44d8-b1e5-f7fc6b54fa34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:41.962806 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b839f6-45f0-44d8-b1e5-f7fc6b54fa34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:41.963500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b839f6-45f0-44d8-b1e5-f7fc6b54fa34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:41.965402 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b839f6-45f0-44d8-b1e5-f7fc6b54fa34 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:41.991208 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1246/2521] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:45:41 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:45:41.995277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ba977-66b3-4286-b957-393cbac07630 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:42.005978 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ba977-66b3-4286-b957-393cbac07630 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:42.066584 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646c0c4-f93b-402f-a9d9-411751dd53f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:42.067492 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646c0c4-f93b-402f-a9d9-411751dd53f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:42.067789 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646c0c4-f93b-402f-a9d9-411751dd53f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:42.068070 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646c0c4-f93b-402f-a9d9-411751dd53f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:42.068786 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646c0c4-f93b-402f-a9d9-411751dd53f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:42.070759 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646c0c4-f93b-402f-a9d9-411751dd53f9 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:42.090887 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1276/2522] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:45:41 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:45:42.095304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b839f6-45f0-44d8-b1e5-f7fc6b54fa34 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:42.106514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b839f6-45f0-44d8-b1e5-f7fc6b54fa34 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:42.111203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139baffd-8b71-4aee-b3c6-4e8d4093601a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:42.111868 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139baffd-8b71-4aee-b3c6-4e8d4093601a admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:42.112158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139baffd-8b71-4aee-b3c6-4e8d4093601a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:42.112441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139baffd-8b71-4aee-b3c6-4e8d4093601a admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:42.113175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139baffd-8b71-4aee-b3c6-4e8d4093601a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:42.114817 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139baffd-8b71-4aee-b3c6-4e8d4093601a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:42.127367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1247/2523] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:45:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:45:42.131364 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646c0c4-f93b-402f-a9d9-411751dd53f9 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:42.141470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646c0c4-f93b-402f-a9d9-411751dd53f9 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:42.146126 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4df98-4f3c-423f-9ede-809ccd721f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:42.146831 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4df98-4f3c-423f-9ede-809ccd721f44 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:42.147118 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4df98-4f3c-423f-9ede-809ccd721f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:42.147399 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4df98-4f3c-423f-9ede-809ccd721f44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:42.148162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c4df98-4f3c-423f-9ede-809ccd721f44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:42.149753 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c4df98-4f3c-423f-9ede-809ccd721f44 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:42.159936 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1277/2524] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:45:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:45:42.164856 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139baffd-8b71-4aee-b3c6-4e8d4093601a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:42.183984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139baffd-8b71-4aee-b3c6-4e8d4093601a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:42.188774 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63294e42-5d7f-41b7-b00f-440356ab00af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:42.189694 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63294e42-5d7f-41b7-b00f-440356ab00af admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:42.189983 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63294e42-5d7f-41b7-b00f-440356ab00af admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:42.190265 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63294e42-5d7f-41b7-b00f-440356ab00af admin admin] PATH_INFO: `/v3/projects/2be8689fa21847ad8239ecef2dd8dd9b/users/707bc0b9491f41a185a132107432e7af/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:42.204945 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63294e42-5d7f-41b7-b00f-440356ab00af admin admin] RBAC: Authorizing `identity:create_grant(project_id=2be8689fa21847ad8239ecef2dd8dd9b, user_id=707bc0b9491f41a185a132107432e7af, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:42.207650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63294e42-5d7f-41b7-b00f-440356ab00af admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:42.220752 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1248/2525] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:45:42 2022] PUT /identity/v3/projects/2be8689fa21847ad8239ecef2dd8dd9b/users/707bc0b9491f41a185a132107432e7af/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:45:42.226273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1448e39-5e07-45d7-9647-2a6efe267f4a None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:42.226810 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1448e39-5e07-45d7-9647-2a6efe267f4a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:42.226926 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1448e39-5e07-45d7-9647-2a6efe267f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:42.244865 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-f1448e39-5e07-45d7-9647-2a6efe267f4a None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:45:42.246071 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f1448e39-5e07-45d7-9647-2a6efe267f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:42.277465 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:42.307552 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1278/2526] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:45:42.317290 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63294e42-5d7f-41b7-b00f-440356ab00af admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:42.319550 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-63294e42-5d7f-41b7-b00f-440356ab00af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:42.379150 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63294e42-5d7f-41b7-b00f-440356ab00af admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:42.383529 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebab8cc-5c03-43b6-992a-b004e290374c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:42.384192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebab8cc-5c03-43b6-992a-b004e290374c service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:42.384482 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebab8cc-5c03-43b6-992a-b004e290374c service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:42.384764 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebab8cc-5c03-43b6-992a-b004e290374c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:42.394090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebab8cc-5c03-43b6-992a-b004e290374c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:42.395560 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebab8cc-5c03-43b6-992a-b004e290374c service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:42.406036 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1249/2527] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:45:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:45:43.050126 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b2ede-ea50-401a-a737-c1687bab9cb3 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:43.050552 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b2ede-ea50-401a-a737-c1687bab9cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:43.050881 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b2ede-ea50-401a-a737-c1687bab9cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:43.081216 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a48b2ede-ea50-401a-a737-c1687bab9cb3 None None] MFA Rules not processed for user `707bc0b9491f41a185a132107432e7af`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:45:43.082576 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a48b2ede-ea50-401a-a737-c1687bab9cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:43.169620 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1279/2528] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:45:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:45:43.181326 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebab8cc-5c03-43b6-992a-b004e290374c service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:45:43.183191 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2ebab8cc-5c03-43b6-992a-b004e290374c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:43.240065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:43.258386 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebab8cc-5c03-43b6-992a-b004e290374c service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:45:43.262684 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c92cf-1dbe-4565-aae8-426f001acf97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:45:43.263419 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c92cf-1dbe-4565-aae8-426f001acf97 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:43.263796 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c92cf-1dbe-4565-aae8-426f001acf97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:43.264116 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c92cf-1dbe-4565-aae8-426f001acf97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:43.276996 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c92cf-1dbe-4565-aae8-426f001acf97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:45:43.278352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c92cf-1dbe-4565-aae8-426f001acf97 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:45:43.289684 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1250/2529] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:45:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:45:44.699808 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd66e2-fe48-4e95-89ea-08c6acc12715 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:44.700340 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd66e2-fe48-4e95-89ea-08c6acc12715 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:44.700340 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd66e2-fe48-4e95-89ea-08c6acc12715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:44.717966 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-9afd66e2-fe48-4e95-89ea-08c6acc12715 None None] MFA Rules not processed for user `707bc0b9491f41a185a132107432e7af`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:45:44.719215 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9afd66e2-fe48-4e95-89ea-08c6acc12715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:44.726107 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1280/2530] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:45:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:45:53.283794 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=6 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:54.712474 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=45 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:45:59.798607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bb185-2b32-4f7d-a783-8c1c3cc068b4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:45:59.799075 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bb185-2b32-4f7d-a783-8c1c3cc068b4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:45:59.799192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bb185-2b32-4f7d-a783-8c1c3cc068b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:45:59.816068 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d37bb185-2b32-4f7d-a783-8c1c3cc068b4 None None] MFA Rules not processed for user `707bc0b9491f41a185a132107432e7af`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:45:59.817339 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d37bb185-2b32-4f7d-a783-8c1c3cc068b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:45:59.824080 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1251/2531] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:45:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 26 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:46:09.811302 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:46:21.877793 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe52324-8aa8-4b32-ab27-c29c11fe4e88 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:46:21.878314 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe52324-8aa8-4b32-ab27-c29c11fe4e88 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:46:21.878445 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe52324-8aa8-4b32-ab27-c29c11fe4e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:46:21.896862 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-cbe52324-8aa8-4b32-ab27-c29c11fe4e88 None None] MFA Rules not processed for user `0b7d100f2407432381d119c25814fff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:46:21.898063 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-cbe52324-8aa8-4b32-ab27-c29c11fe4e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:46:21.955373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1281/2532] 10.222.0.67 () {58 vars in 1009 bytes} [Tue May 31 14:46:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:46:22.000295 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bb185-2b32-4f7d-a783-8c1c3cc068b4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:46:22.001703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d37bb185-2b32-4f7d-a783-8c1c3cc068b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:46:22.072643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bb185-2b32-4f7d-a783-8c1c3cc068b4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:46:22.077952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d99de6-788a-4579-a327-bb33f8240250 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfa2282f957941b98db1d7497da3351b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:46:22.078654 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d99de6-788a-4579-a327-bb33f8240250 service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:46:22.078964 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d99de6-788a-4579-a327-bb33f8240250 service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:46:22.079248 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d99de6-788a-4579-a327-bb33f8240250 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:46:22.087889 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d99de6-788a-4579-a327-bb33f8240250 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:46:22.089671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d99de6-788a-4579-a327-bb33f8240250 service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:46:22.098440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1252/2533] 10.222.0.67 () {60 vars in 1384 bytes} [Tue May 31 14:46:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 31 14:46:31.947830 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=25 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:46:32.094886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=32 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:46:36.383226 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4b1e5-e6a8-4f92-a28a-781dffd8c2ff None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:46:36.383706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4b1e5-e6a8-4f92-a28a-781dffd8c2ff None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:46:36.383812 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4b1e5-e6a8-4f92-a28a-781dffd8c2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:46:36.401489 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a7a4b1e5-e6a8-4f92-a28a-781dffd8c2ff None None] MFA Rules not processed for user `0b7d100f2407432381d119c25814fff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:46:36.402708 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a7a4b1e5-e6a8-4f92-a28a-781dffd8c2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:46:36.409659 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1282/2534] 10.222.0.67 () {58 vars in 967 bytes} [Tue May 31 14:46:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:46:36.421854 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d99de6-788a-4579-a327-bb33f8240250 service swift] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:46:36.432514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d99de6-788a-4579-a327-bb33f8240250 service swift] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:46:36.437382 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd924f-a866-41eb-be1a-17d540e1474e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfa2282f957941b98db1d7497da3351b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:46:36.438018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd924f-a866-41eb-be1a-17d540e1474e service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:46:36.438300 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd924f-a866-41eb-be1a-17d540e1474e service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:46:36.438574 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd924f-a866-41eb-be1a-17d540e1474e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:46:36.447034 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd924f-a866-41eb-be1a-17d540e1474e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:46:36.448588 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd924f-a866-41eb-be1a-17d540e1474e service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:46:36.456476 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1253/2535] 10.222.0.67 () {60 vars in 1384 bytes} [Tue May 31 14:46:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 31 14:46:38.318923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4b1e5-e6a8-4f92-a28a-781dffd8c2ff None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:46:38.331965 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4b1e5-e6a8-4f92-a28a-781dffd8c2ff None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:46:38.336578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5603b4-83a4-40ec-8db2-65bd2ee5af0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:46:38.337454 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5603b4-83a4-40ec-8db2-65bd2ee5af0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:46:38.337741 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5603b4-83a4-40ec-8db2-65bd2ee5af0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:46:38.338024 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5603b4-83a4-40ec-8db2-65bd2ee5af0b admin admin] PATH_INFO: `/v3/users/707bc0b9491f41a185a132107432e7af` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:46:38.339246 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5603b4-83a4-40ec-8db2-65bd2ee5af0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=707bc0b9491f41a185a132107432e7af)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:46:38.341173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5603b4-83a4-40ec-8db2-65bd2ee5af0b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:46:38.388487 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3b5603b4-83a4-40ec-8db2-65bd2ee5af0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '707bc0b9491f41a185a132107432e7af'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:46:38.395878 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3b5603b4-83a4-40ec-8db2-65bd2ee5af0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '707bc0b9491f41a185a132107432e7af'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:46:38.396988 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3b5603b4-83a4-40ec-8db2-65bd2ee5af0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '707bc0b9491f41a185a132107432e7af'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:46:38.408914 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3b5603b4-83a4-40ec-8db2-65bd2ee5af0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '707bc0b9491f41a185a132107432e7af'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:46:38.409552 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-3b5603b4-83a4-40ec-8db2-65bd2ee5af0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '707bc0b9491f41a185a132107432e7af'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:46:38.421649 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1283/2536] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 14:46:38 2022] DELETE /identity/v3/users/707bc0b9491f41a185a132107432e7af => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:46:38.598515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd924f-a866-41eb-be1a-17d540e1474e service swift] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:46:38.610674 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd924f-a866-41eb-be1a-17d540e1474e service swift] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:46:38.668091 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4067a-e06e-4329-a836-748f9e44e96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:46:38.668948 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4067a-e06e-4329-a836-748f9e44e96b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:46:38.669228 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4067a-e06e-4329-a836-748f9e44e96b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:46:38.669500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4067a-e06e-4329-a836-748f9e44e96b admin admin] PATH_INFO: `/v3/projects/2be8689fa21847ad8239ecef2dd8dd9b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:46:38.670683 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4067a-e06e-4329-a836-748f9e44e96b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2be8689fa21847ad8239ecef2dd8dd9b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:46:38.672617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4067a-e06e-4329-a836-748f9e44e96b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:46:38.718841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-adb4067a-e06e-4329-a836-748f9e44e96b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2be8689fa21847ad8239ecef2dd8dd9b', 'request_id': 'req-adb4067a-e06e-4329-a836-748f9e44e96b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:46:38.724222 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-adb4067a-e06e-4329-a836-748f9e44e96b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2be8689fa21847ad8239ecef2dd8dd9b', 'request_id': 'req-adb4067a-e06e-4329-a836-748f9e44e96b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:46:38.730865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-adb4067a-e06e-4329-a836-748f9e44e96b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2be8689fa21847ad8239ecef2dd8dd9b', 'request_id': 'req-adb4067a-e06e-4329-a836-748f9e44e96b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:46:38.743941 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-adb4067a-e06e-4329-a836-748f9e44e96b admin admin] The token cache is being invalidate because project 2be8689fa21847ad8239ecef2dd8dd9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 14:46:38.744231 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-adb4067a-e06e-4329-a836-748f9e44e96b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:46:38.744852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-adb4067a-e06e-4329-a836-748f9e44e96b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 14:46:38.746373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1254/2537] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 14:46:38 2022] DELETE /identity/v3/projects/2be8689fa21847ad8239ecef2dd8dd9b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:46:38.751446 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d7857-33b1-4e87-b955-ade255247621 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:46:38.751811 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d7857-33b1-4e87-b955-ade255247621 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:46:38.752157 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d7857-33b1-4e87-b955-ade255247621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:46:38.771078 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-167d7857-33b1-4e87-b955-ade255247621 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:46:38.772297 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-167d7857-33b1-4e87-b955-ade255247621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:46:38.830370 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1284/2538] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:46:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:46:38.890038 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f67558-ee76-48e1-8686-20d1f90c6093 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:46:38.890311 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f67558-ee76-48e1-8686-20d1f90c6093 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:46:38.890590 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f67558-ee76-48e1-8686-20d1f90c6093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:46:38.909084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-73f67558-ee76-48e1-8686-20d1f90c6093 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:46:38.910304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-73f67558-ee76-48e1-8686-20d1f90c6093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:46:38.918861 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1255/2539] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:46:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:46:38.923110 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d7857-33b1-4e87-b955-ade255247621 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:46:38.934666 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d7857-33b1-4e87-b955-ade255247621 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:46:38.939310 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c5f70-096e-4d34-8f8b-e49a71ac6e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:46:38.940015 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c5f70-096e-4d34-8f8b-e49a71ac6e95 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:46:38.940284 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c5f70-096e-4d34-8f8b-e49a71ac6e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:46:38.940559 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c5f70-096e-4d34-8f8b-e49a71ac6e95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:46:38.941337 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c5f70-096e-4d34-8f8b-e49a71ac6e95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 14:46:38.941642 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c5f70-096e-4d34-8f8b-e49a71ac6e95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:46:38.943292 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c5f70-096e-4d34-8f8b-e49a71ac6e95 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:46:38.955273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1285/2540] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 14:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 14:46:38.959609 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f67558-ee76-48e1-8686-20d1f90c6093 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:46:38.968932 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f67558-ee76-48e1-8686-20d1f90c6093 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:46:38.973566 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c96f60-8fb2-4612-9b4c-7d42429fd527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:46:38.974241 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c96f60-8fb2-4612-9b4c-7d42429fd527 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:46:38.974569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c96f60-8fb2-4612-9b4c-7d42429fd527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:46:38.974826 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c96f60-8fb2-4612-9b4c-7d42429fd527 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:46:38.976007 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c96f60-8fb2-4612-9b4c-7d42429fd527 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:46:38.977891 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c96f60-8fb2-4612-9b4c-7d42429fd527 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:46:39.003669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1256/2541] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 14:46:38 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:46:39.007823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c5f70-096e-4d34-8f8b-e49a71ac6e95 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:46:39.017911 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c5f70-096e-4d34-8f8b-e49a71ac6e95 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:46:39.065302 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:46:39.075530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a705a4a-1709-49a7-9c2a-c717108ec992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:46:39.076420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a705a4a-1709-49a7-9c2a-c717108ec992 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:46:39.076889 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a705a4a-1709-49a7-9c2a-c717108ec992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:46:39.077180 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a705a4a-1709-49a7-9c2a-c717108ec992 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:46:39.078073 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a705a4a-1709-49a7-9c2a-c717108ec992 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:46:39.080152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a705a4a-1709-49a7-9c2a-c717108ec992 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:46:39.100156 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1286/2542] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 14:46:39 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 14:46:39.104983 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c96f60-8fb2-4612-9b4c-7d42429fd527 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:46:39.115350 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c96f60-8fb2-4612-9b4c-7d42429fd527 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:46:39.119993 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6436d9f-f8f3-4867-836b-309b249b8263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:46:39.120685 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6436d9f-f8f3-4867-836b-309b249b8263 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:46:39.120953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6436d9f-f8f3-4867-836b-309b249b8263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:46:39.121235 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6436d9f-f8f3-4867-836b-309b249b8263 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:46:39.121965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6436d9f-f8f3-4867-836b-309b249b8263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:46:39.123601 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6436d9f-f8f3-4867-836b-309b249b8263 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:46:39.134426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1257/2543] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:46:39.138356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a705a4a-1709-49a7-9c2a-c717108ec992 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:46:39.149608 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a705a4a-1709-49a7-9c2a-c717108ec992 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:46:39.154604 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249306b8-9737-4eef-b855-4def3ef0b012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:46:39.155296 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249306b8-9737-4eef-b855-4def3ef0b012 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:46:39.155581 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249306b8-9737-4eef-b855-4def3ef0b012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:46:39.155863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249306b8-9737-4eef-b855-4def3ef0b012 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:46:39.156593 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249306b8-9737-4eef-b855-4def3ef0b012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:46:39.158193 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249306b8-9737-4eef-b855-4def3ef0b012 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:46:39.168954 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1287/2544] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 14:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 14:46:39.178005 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6436d9f-f8f3-4867-836b-309b249b8263 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:46:39.188271 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6436d9f-f8f3-4867-836b-309b249b8263 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:46:39.192891 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc6138-4a94-4415-b6ec-224a4e935d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:46:39.193772 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc6138-4a94-4415-b6ec-224a4e935d2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:46:39.194093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc6138-4a94-4415-b6ec-224a4e935d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:46:39.194412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc6138-4a94-4415-b6ec-224a4e935d2f admin admin] PATH_INFO: `/v3/projects/243bbd81047e44acb2e0110e4002eb00/users/3375ea629b5f4c20a0331a610a1712e1/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:46:39.208652 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc6138-4a94-4415-b6ec-224a4e935d2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=243bbd81047e44acb2e0110e4002eb00, user_id=3375ea629b5f4c20a0331a610a1712e1, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:46:39.211819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc6138-4a94-4415-b6ec-224a4e935d2f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:46:39.224347 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1258/2545] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 14:46:39 2022] PUT /identity/v3/projects/243bbd81047e44acb2e0110e4002eb00/users/3375ea629b5f4c20a0331a610a1712e1/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 14:46:39.234705 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4019e76-97e6-477f-8899-079b043d40f1 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:46:39.235003 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4019e76-97e6-477f-8899-079b043d40f1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:46:39.235294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4019e76-97e6-477f-8899-079b043d40f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:46:39.263222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-e4019e76-97e6-477f-8899-079b043d40f1 None None] MFA Rules not processed for user `3375ea629b5f4c20a0331a610a1712e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:46:39.264460 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e4019e76-97e6-477f-8899-079b043d40f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:46:39.329359 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1288/2546] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 14:46:39 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:46:39.340836 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc6138-4a94-4415-b6ec-224a4e935d2f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:46:39.342535 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-22fc6138-4a94-4415-b6ec-224a4e935d2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:46:39.357202 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:46:39.401706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc6138-4a94-4415-b6ec-224a4e935d2f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:46:39.405353 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7834a16-b45a-4269-8dd1-5d3fb9b750ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:46:39.406003 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7834a16-b45a-4269-8dd1-5d3fb9b750ce service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:46:39.406369 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7834a16-b45a-4269-8dd1-5d3fb9b750ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:46:39.406673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7834a16-b45a-4269-8dd1-5d3fb9b750ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:46:39.415491 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7834a16-b45a-4269-8dd1-5d3fb9b750ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:46:39.416923 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7834a16-b45a-4269-8dd1-5d3fb9b750ce service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:46:39.425668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1259/2547] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:46:39 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:46:44.012112 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4019e76-97e6-477f-8899-079b043d40f1 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:46:44.013515 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e4019e76-97e6-477f-8899-079b043d40f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:46:44.084468 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4019e76-97e6-477f-8899-079b043d40f1 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:46:44.090080 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92781162-1536-47b5-80b0-7ec8b22bbab8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c1c13515a8b41a781478dc65e6a88b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:46:44.090754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92781162-1536-47b5-80b0-7ec8b22bbab8 service placement] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:46:44.091037 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92781162-1536-47b5-80b0-7ec8b22bbab8 service placement] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:46:44.091312 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92781162-1536-47b5-80b0-7ec8b22bbab8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:46:44.093123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-92781162-1536-47b5-80b0-7ec8b22bbab8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:46:44.104480 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92781162-1536-47b5-80b0-7ec8b22bbab8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:46:44.106055 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92781162-1536-47b5-80b0-7ec8b22bbab8 service placement] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:46:44.115758 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1289/2548] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:46:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:46:49.421847 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=20 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:46:54.111668 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=74,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:47:17.917701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d1d49-2561-4316-9125-fd75f7e0eda6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:47:17.918274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d1d49-2561-4316-9125-fd75f7e0eda6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:47:17.918274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d1d49-2561-4316-9125-fd75f7e0eda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:47:17.938846 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f19d1d49-2561-4316-9125-fd75f7e0eda6 None None] MFA Rules not processed for user `0b7d100f2407432381d119c25814fff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:47:17.940352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f19d1d49-2561-4316-9125-fd75f7e0eda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:47:18.007348 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1260/2549] 10.222.0.67 () {58 vars in 1009 bytes} [Tue May 31 14:47:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:47:18.027552 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92781162-1536-47b5-80b0-7ec8b22bbab8 service placement] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:47:18.030943 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-92781162-1536-47b5-80b0-7ec8b22bbab8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:47:18.122984 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92781162-1536-47b5-80b0-7ec8b22bbab8 service placement] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:47:18.128586 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f1976-e129-48b8-950e-648289e90020 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfa2282f957941b98db1d7497da3351b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:47:18.128923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f1976-e129-48b8-950e-648289e90020 service swift] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:47:18.129211 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f1976-e129-48b8-950e-648289e90020 service swift] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:47:18.129627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f1976-e129-48b8-950e-648289e90020 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:47:18.138580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5f1976-e129-48b8-950e-648289e90020 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:47:18.140226 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5f1976-e129-48b8-950e-648289e90020 service swift] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:47:18.149043 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1290/2550] 10.222.0.67 () {60 vars in 1384 bytes} [Tue May 31 14:47:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 31 14:47:20.139158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d1d49-2561-4316-9125-fd75f7e0eda6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:47:20.140581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f19d1d49-2561-4316-9125-fd75f7e0eda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:47:20.161599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d1d49-2561-4316-9125-fd75f7e0eda6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:47:20.167683 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2dcb66-6c3e-4758-974a-d07bf333d525 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:47:20.168512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2dcb66-6c3e-4758-974a-d07bf333d525 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:47:20.169144 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2dcb66-6c3e-4758-974a-d07bf333d525 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:47:20.169144 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2dcb66-6c3e-4758-974a-d07bf333d525 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:47:20.171207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7d2dcb66-6c3e-4758-974a-d07bf333d525 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:47:20.247635 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2dcb66-6c3e-4758-974a-d07bf333d525 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:47:20.249538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2dcb66-6c3e-4758-974a-d07bf333d525 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:47:20.264964 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1261/2551] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:47:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:47:28.145409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=32 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:47:30.256803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=53 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:48:16.903957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd80dc-8a6b-4f8f-82c6-b98149e9f354 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:48:16.904500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd80dc-8a6b-4f8f-82c6-b98149e9f354 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:48:16.904500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd80dc-8a6b-4f8f-82c6-b98149e9f354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:48:16.922100 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-63fd80dc-8a6b-4f8f-82c6-b98149e9f354 None None] MFA Rules not processed for user `0b7d100f2407432381d119c25814fff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:48:16.923526 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-63fd80dc-8a6b-4f8f-82c6-b98149e9f354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:48:16.931241 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1291/2552] 10.222.0.67 () {58 vars in 1009 bytes} [Tue May 31 14:48:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:48:16.943146 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2dcb66-6c3e-4758-974a-d07bf333d525 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:48:16.957288 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2dcb66-6c3e-4758-974a-d07bf333d525 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:48:16.962242 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167ef90-78f5-4efa-a65e-9c825bc7d799 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfa2282f957941b98db1d7497da3351b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:48:16.962940 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167ef90-78f5-4efa-a65e-9c825bc7d799 service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:48:16.963230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167ef90-78f5-4efa-a65e-9c825bc7d799 service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:48:16.963512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167ef90-78f5-4efa-a65e-9c825bc7d799 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:48:16.972606 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167ef90-78f5-4efa-a65e-9c825bc7d799 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:48:16.974210 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167ef90-78f5-4efa-a65e-9c825bc7d799 service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:48:16.983467 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1262/2553] 10.222.0.67 () {60 vars in 1384 bytes} [Tue May 31 14:48:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 31 14:48:26.917061 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:48:26.979723 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=6 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:48:34.301174 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd80dc-8a6b-4f8f-82c6-b98149e9f354 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:48:34.302569 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-63fd80dc-8a6b-4f8f-82c6-b98149e9f354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:48:34.359205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd80dc-8a6b-4f8f-82c6-b98149e9f354 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:48:34.365449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7b543-9da4-4c50-859f-99dad2af7c22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:48:34.366096 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7b543-9da4-4c50-859f-99dad2af7c22 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:48:34.366382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7b543-9da4-4c50-859f-99dad2af7c22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:48:34.366680 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7b543-9da4-4c50-859f-99dad2af7c22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:48:34.368453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-84a7b543-9da4-4c50-859f-99dad2af7c22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:48:34.439018 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a7b543-9da4-4c50-859f-99dad2af7c22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:48:34.440387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a7b543-9da4-4c50-859f-99dad2af7c22 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:48:34.452074 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1292/2554] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:48:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:48:36.717014 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939afc4-d20c-4cb8-a37a-d968949ed984 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:48:36.717469 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939afc4-d20c-4cb8-a37a-d968949ed984 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:48:36.717578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939afc4-d20c-4cb8-a37a-d968949ed984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:48:36.735284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a939afc4-d20c-4cb8-a37a-d968949ed984 None None] MFA Rules not processed for user `0b7d100f2407432381d119c25814fff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 14:48:36.736493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a939afc4-d20c-4cb8-a37a-d968949ed984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:48:36.743683 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1263/2555] 10.222.0.67 () {58 vars in 967 bytes} [Tue May 31 14:48:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 14:48:36.764614 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7b543-9da4-4c50-859f-99dad2af7c22 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:48:36.776058 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7b543-9da4-4c50-859f-99dad2af7c22 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:48:36.781560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b1525-f1ec-4247-842a-73e8f3f73da1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfa2282f957941b98db1d7497da3351b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:48:36.782419 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b1525-f1ec-4247-842a-73e8f3f73da1 service swift] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:48:36.782734 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b1525-f1ec-4247-842a-73e8f3f73da1 service swift] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:48:36.783013 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b1525-f1ec-4247-842a-73e8f3f73da1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:48:36.791216 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b1525-f1ec-4247-842a-73e8f3f73da1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:48:36.792789 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b1525-f1ec-4247-842a-73e8f3f73da1 service swift] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:48:36.801758 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1293/2556] 10.222.0.67 () {60 vars in 1384 bytes} [Tue May 31 14:48:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 31 14:48:36.809395 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939afc4-d20c-4cb8-a37a-d968949ed984 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:48:36.819705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939afc4-d20c-4cb8-a37a-d968949ed984 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:48:36.823397 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8affc17-1941-456b-bc4d-41a1d1ecd3e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:48:36.824096 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8affc17-1941-456b-bc4d-41a1d1ecd3e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:48:36.824325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8affc17-1941-456b-bc4d-41a1d1ecd3e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:48:36.824605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8affc17-1941-456b-bc4d-41a1d1ecd3e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:48:36.826541 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a8affc17-1941-456b-bc4d-41a1d1ecd3e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:48:36.841128 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8affc17-1941-456b-bc4d-41a1d1ecd3e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:48:36.842943 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8affc17-1941-456b-bc4d-41a1d1ecd3e2 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:48:36.854080 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1264/2557] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:48:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:48:44.734116 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b1525-f1ec-4247-842a-73e8f3f73da1 service swift] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:48:44.743671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b1525-f1ec-4247-842a-73e8f3f73da1 service swift] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:48:44.747396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b333c-8675-4289-8363-edd3e1ac9f0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:48:44.748041 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b333c-8675-4289-8363-edd3e1ac9f0f service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:48:44.748329 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b333c-8675-4289-8363-edd3e1ac9f0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:48:44.748606 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b333c-8675-4289-8363-edd3e1ac9f0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:48:44.750412 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-948b333c-8675-4289-8363-edd3e1ac9f0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:48:44.837452 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948b333c-8675-4289-8363-edd3e1ac9f0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:48:44.838945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948b333c-8675-4289-8363-edd3e1ac9f0f service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:48:44.848884 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1294/2558] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:48:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:48:46.849678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=9 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:48:54.844433 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:49:45.155492 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8affc17-1941-456b-bc4d-41a1d1ecd3e2 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:49:45.157050 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a8affc17-1941-456b-bc4d-41a1d1ecd3e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:49:45.169888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8affc17-1941-456b-bc4d-41a1d1ecd3e2 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:49:45.174013 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836e95e-81ec-44a2-8939-8b5592a56434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:49:45.174829 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836e95e-81ec-44a2-8939-8b5592a56434 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:49:45.175001 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836e95e-81ec-44a2-8939-8b5592a56434 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:49:45.175282 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836e95e-81ec-44a2-8939-8b5592a56434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:49:45.177157 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b836e95e-81ec-44a2-8939-8b5592a56434 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:49:45.188430 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b836e95e-81ec-44a2-8939-8b5592a56434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:49:45.189855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b836e95e-81ec-44a2-8939-8b5592a56434 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:49:45.198046 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1265/2559] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:49:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:49:55.194397 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=6 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:50:18.832463 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b333c-8675-4289-8363-edd3e1ac9f0f service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:50:18.841723 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b333c-8675-4289-8363-edd3e1ac9f0f service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:50:18.847509 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44d882-2b53-48c4-b9f9-a0ea26b9d7c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:50:18.848416 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44d882-2b53-48c4-b9f9-a0ea26b9d7c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:50:18.848855 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44d882-2b53-48c4-b9f9-a0ea26b9d7c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:50:18.849284 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44d882-2b53-48c4-b9f9-a0ea26b9d7c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:50:18.851572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ff44d882-2b53-48c4-b9f9-a0ea26b9d7c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:50:18.948052 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44d882-2b53-48c4-b9f9-a0ea26b9d7c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:50:18.949426 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44d882-2b53-48c4-b9f9-a0ea26b9d7c5 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:50:18.977168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1295/2560] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:50:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:50:28.971408 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=41 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:51:39.850714 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836e95e-81ec-44a2-8939-8b5592a56434 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:51:39.870431 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836e95e-81ec-44a2-8939-8b5592a56434 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:51:39.874098 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2f62c-f0fb-4a95-ac87-d95109b8f664 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:51:39.874777 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2f62c-f0fb-4a95-ac87-d95109b8f664 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:51:39.875068 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2f62c-f0fb-4a95-ac87-d95109b8f664 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:51:39.875350 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2f62c-f0fb-4a95-ac87-d95109b8f664 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:51:39.897883 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2f62c-f0fb-4a95-ac87-d95109b8f664 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:51:39.899345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2f62c-f0fb-4a95-ac87-d95109b8f664 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:51:39.908075 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1266/2561] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:51:39 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:51:48.409144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44d882-2b53-48c4-b9f9-a0ea26b9d7c5 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:51:48.411073 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ff44d882-2b53-48c4-b9f9-a0ea26b9d7c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:51:48.501206 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44d882-2b53-48c4-b9f9-a0ea26b9d7c5 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:51:48.506465 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcffc4-9d6b-4b83-8baa-1e31fbbf7713 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c1c13515a8b41a781478dc65e6a88b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:51:48.507155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcffc4-9d6b-4b83-8baa-1e31fbbf7713 service placement] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:51:48.507475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcffc4-9d6b-4b83-8baa-1e31fbbf7713 service placement] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:51:48.507758 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcffc4-9d6b-4b83-8baa-1e31fbbf7713 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:51:48.516430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcffc4-9d6b-4b83-8baa-1e31fbbf7713 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:51:48.518027 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcffc4-9d6b-4b83-8baa-1e31fbbf7713 service placement] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:51:48.527996 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1296/2562] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:51:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:51:49.904134 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=17 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:51:58.523735 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=37 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:52:19.889665 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2f62c-f0fb-4a95-ac87-d95109b8f664 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:52:19.899110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2f62c-f0fb-4a95-ac87-d95109b8f664 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:52:19.902782 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed98fd-b4b2-4e71-a6ca-c3d777f00835 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:52:19.903435 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed98fd-b4b2-4e71-a6ca-c3d777f00835 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:52:19.903724 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed98fd-b4b2-4e71-a6ca-c3d777f00835 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:52:19.904004 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed98fd-b4b2-4e71-a6ca-c3d777f00835 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:52:19.912635 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed98fd-b4b2-4e71-a6ca-c3d777f00835 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:52:19.914050 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed98fd-b4b2-4e71-a6ca-c3d777f00835 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:52:19.941540 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1267/2563] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:52:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:52:29.938357 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=13 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:53:45.263269 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcffc4-9d6b-4b83-8baa-1e31fbbf7713 service placement] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:53:45.282751 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcffc4-9d6b-4b83-8baa-1e31fbbf7713 service placement] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:53:45.286311 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ad972-f5f8-4a5b-81ab-9efe735239b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:53:45.286987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ad972-f5f8-4a5b-81ab-9efe735239b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:53:45.287276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ad972-f5f8-4a5b-81ab-9efe735239b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:53:45.287556 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ad972-f5f8-4a5b-81ab-9efe735239b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:53:45.296705 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ad972-f5f8-4a5b-81ab-9efe735239b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:53:45.298133 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ad972-f5f8-4a5b-81ab-9efe735239b0 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:53:45.317446 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1297/2564] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:53:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:53:55.314879 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=14 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:54:11.807294 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed98fd-b4b2-4e71-a6ca-c3d777f00835 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:54:11.825755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed98fd-b4b2-4e71-a6ca-c3d777f00835 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:54:11.829342 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e3d99-b469-4237-8daf-b75b7a6ce0ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:54:11.829974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e3d99-b469-4237-8daf-b75b7a6ce0ea service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:54:11.830255 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e3d99-b469-4237-8daf-b75b7a6ce0ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:54:11.830529 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e3d99-b469-4237-8daf-b75b7a6ce0ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:54:11.847056 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e3d99-b469-4237-8daf-b75b7a6ce0ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:54:11.848461 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e3d99-b469-4237-8daf-b75b7a6ce0ea service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:54:11.869102 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1268/2565] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:54:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:54:21.866511 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=17 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:54:47.081568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ad972-f5f8-4a5b-81ab-9efe735239b0 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:54:47.102734 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ad972-f5f8-4a5b-81ab-9efe735239b0 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:54:47.106357 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9fb84-b7a2-4e90-8bc3-1e0a742dce20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:54:47.107029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9fb84-b7a2-4e90-8bc3-1e0a742dce20 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:54:47.107318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9fb84-b7a2-4e90-8bc3-1e0a742dce20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:54:47.107596 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9fb84-b7a2-4e90-8bc3-1e0a742dce20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:54:47.115023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f9fb84-b7a2-4e90-8bc3-1e0a742dce20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:54:47.116452 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f9fb84-b7a2-4e90-8bc3-1e0a742dce20 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:54:47.124741 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1298/2566] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:54:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:54:57.120912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=10 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:55:19.785370 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e3d99-b469-4237-8daf-b75b7a6ce0ea service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:55:19.795816 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e3d99-b469-4237-8daf-b75b7a6ce0ea service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:55:19.800318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff02d49-3344-4a45-908d-ac45870920e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:55:19.800983 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff02d49-3344-4a45-908d-ac45870920e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:55:19.801270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff02d49-3344-4a45-908d-ac45870920e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:55:19.801550 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff02d49-3344-4a45-908d-ac45870920e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:55:19.811610 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff02d49-3344-4a45-908d-ac45870920e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:55:19.813048 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff02d49-3344-4a45-908d-ac45870920e3 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:55:19.822487 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1269/2567] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:55:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:55:29.818174 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=6 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:56:38.851447 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9fb84-b7a2-4e90-8bc3-1e0a742dce20 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:56:38.861144 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9fb84-b7a2-4e90-8bc3-1e0a742dce20 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:56:38.864712 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75cd84-f557-4e7f-a86a-89b033fb9b43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:56:38.865350 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75cd84-f557-4e7f-a86a-89b033fb9b43 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:56:38.865634 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75cd84-f557-4e7f-a86a-89b033fb9b43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:56:38.865916 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75cd84-f557-4e7f-a86a-89b033fb9b43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:56:38.874032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75cd84-f557-4e7f-a86a-89b033fb9b43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:56:38.875494 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75cd84-f557-4e7f-a86a-89b033fb9b43 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:56:38.884206 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1299/2568] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:56:38 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:56:48.880233 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=6 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:56:55.304952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff02d49-3344-4a45-908d-ac45870920e3 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:56:55.306458 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3ff02d49-3344-4a45-908d-ac45870920e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:56:55.373877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff02d49-3344-4a45-908d-ac45870920e3 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:56:55.379797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e4b84-89e6-4272-b35c-4d996c269215 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c1c13515a8b41a781478dc65e6a88b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:56:55.380459 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e4b84-89e6-4272-b35c-4d996c269215 service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:56:55.380774 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e4b84-89e6-4272-b35c-4d996c269215 service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:56:55.381057 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e4b84-89e6-4272-b35c-4d996c269215 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:56:55.383029 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cc2e4b84-89e6-4272-b35c-4d996c269215 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:56:55.394742 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e4b84-89e6-4272-b35c-4d996c269215 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:56:55.396354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e4b84-89e6-4272-b35c-4d996c269215 service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:56:55.406519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1270/2569] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:56:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:57:05.402211 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=32 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:57:19.945567 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75cd84-f557-4e7f-a86a-89b033fb9b43 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:57:19.947200 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-de75cd84-f557-4e7f-a86a-89b033fb9b43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:57:20.022696 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75cd84-f557-4e7f-a86a-89b033fb9b43 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:57:20.026570 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd2411-ea02-49b2-a024-e090332b09b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:57:20.027412 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd2411-ea02-49b2-a024-e090332b09b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:57:20.027814 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd2411-ea02-49b2-a024-e090332b09b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:57:20.028213 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd2411-ea02-49b2-a024-e090332b09b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:57:20.038943 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd2411-ea02-49b2-a024-e090332b09b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:57:20.040566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd2411-ea02-49b2-a024-e090332b09b7 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:57:20.050927 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1300/2570] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:57:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:57:30.045945 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=31 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:58:45.723090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e4b84-89e6-4272-b35c-4d996c269215 service placement] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:58:45.724637 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cc2e4b84-89e6-4272-b35c-4d996c269215 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:58:45.836061 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e4b84-89e6-4272-b35c-4d996c269215 service placement] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:58:45.843882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8b7e5-7222-45ca-9548-6d05f874aab7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:58:45.845094 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8b7e5-7222-45ca-9548-6d05f874aab7 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:58:45.845676 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8b7e5-7222-45ca-9548-6d05f874aab7 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:58:45.846278 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8b7e5-7222-45ca-9548-6d05f874aab7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:58:45.849816 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2fd8b7e5-7222-45ca-9548-6d05f874aab7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:58:45.942465 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd8b7e5-7222-45ca-9548-6d05f874aab7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:58:45.944116 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd8b7e5-7222-45ca-9548-6d05f874aab7 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:58:45.958669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1271/2571] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:58:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:58:55.949388 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=70 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:59:22.803736 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd2411-ea02-49b2-a024-e090332b09b7 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:59:22.818163 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd2411-ea02-49b2-a024-e090332b09b7 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:59:22.827166 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59146161-7724-4bd7-b5b3-b338a42ce86a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:59:22.827850 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59146161-7724-4bd7-b5b3-b338a42ce86a service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:59:22.828141 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59146161-7724-4bd7-b5b3-b338a42ce86a service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:59:22.828501 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59146161-7724-4bd7-b5b3-b338a42ce86a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:59:22.830685 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-59146161-7724-4bd7-b5b3-b338a42ce86a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:59:22.854384 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59146161-7724-4bd7-b5b3-b338a42ce86a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:59:22.861777 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59146161-7724-4bd7-b5b3-b338a42ce86a service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:59:22.883509 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1301/2572] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:59:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:59:23.036148 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8b7e5-7222-45ca-9548-6d05f874aab7 service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:59:23.038013 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2fd8b7e5-7222-45ca-9548-6d05f874aab7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:59:23.133360 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8b7e5-7222-45ca-9548-6d05f874aab7 service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:59:23.143418 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322c368-123e-49b5-b632-931506828a64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:59:23.144393 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322c368-123e-49b5-b632-931506828a64 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:59:23.144760 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322c368-123e-49b5-b632-931506828a64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:59:23.145149 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322c368-123e-49b5-b632-931506828a64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:59:23.148333 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b322c368-123e-49b5-b632-931506828a64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:59:23.171956 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322c368-123e-49b5-b632-931506828a64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:59:23.177401 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322c368-123e-49b5-b632-931506828a64 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:59:23.202391 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1272/2573] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:59:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 14:59:25.522532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59146161-7724-4bd7-b5b3-b338a42ce86a service nova] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:59:25.532706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59146161-7724-4bd7-b5b3-b338a42ce86a service nova] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:59:25.536259 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4f65d-d783-443e-bc06-167cd5ca6e4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:59:25.536909 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4f65d-d783-443e-bc06-167cd5ca6e4b service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:59:25.537187 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4f65d-d783-443e-bc06-167cd5ca6e4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:59:25.537460 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4f65d-d783-443e-bc06-167cd5ca6e4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:59:25.539403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-83d4f65d-d783-443e-bc06-167cd5ca6e4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:59:25.554157 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d4f65d-d783-443e-bc06-167cd5ca6e4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:59:25.555638 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d4f65d-d783-443e-bc06-167cd5ca6e4b service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:59:25.567209 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1302/2574] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:59:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 14:59:33.189665 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=31 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:59:35.561896 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=12 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 14:59:54.120516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322c368-123e-49b5-b632-931506828a64 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 14:59:54.123193 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b322c368-123e-49b5-b632-931506828a64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:59:54.136352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322c368-123e-49b5-b632-931506828a64 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 14:59:54.140703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a585f-e068-4bb5-9807-352185b94a82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 14:59:54.141521 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a585f-e068-4bb5-9807-352185b94a82 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 14:59:54.141907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a585f-e068-4bb5-9807-352185b94a82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 14:59:54.142289 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a585f-e068-4bb5-9807-352185b94a82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 14:59:54.144691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d14a585f-e068-4bb5-9807-352185b94a82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 14:59:54.159855 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a585f-e068-4bb5-9807-352185b94a82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 14:59:54.162584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a585f-e068-4bb5-9807-352185b94a82 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 14:59:54.179699 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1273/2575] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 14:59:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 15:00:04.172210 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=6 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:01:24.585285 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4f65d-d783-443e-bc06-167cd5ca6e4b service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:01:24.595288 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4f65d-d783-443e-bc06-167cd5ca6e4b service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:01:24.598886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c49741-0482-4301-870d-c5606dce47ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:01:24.599527 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c49741-0482-4301-870d-c5606dce47ad service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:01:24.599812 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c49741-0482-4301-870d-c5606dce47ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:01:24.600090 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c49741-0482-4301-870d-c5606dce47ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:01:24.601967 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-31c49741-0482-4301-870d-c5606dce47ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:01:24.682611 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c49741-0482-4301-870d-c5606dce47ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:01:24.684078 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c49741-0482-4301-870d-c5606dce47ad service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:01:24.702857 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1303/2576] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:01:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:01:34.700094 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=41 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:01:58.306924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a585f-e068-4bb5-9807-352185b94a82 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:01:58.308644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d14a585f-e068-4bb5-9807-352185b94a82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:01:58.406235 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a585f-e068-4bb5-9807-352185b94a82 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:01:58.411713 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aefac79-b130-4158-8e65-84c5209c6bdc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c1c13515a8b41a781478dc65e6a88b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:01:58.412400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aefac79-b130-4158-8e65-84c5209c6bdc service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:01:58.412684 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aefac79-b130-4158-8e65-84c5209c6bdc service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:01:58.413068 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aefac79-b130-4158-8e65-84c5209c6bdc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:01:58.424595 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aefac79-b130-4158-8e65-84c5209c6bdc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:01:58.426203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aefac79-b130-4158-8e65-84c5209c6bdc service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:01:58.437487 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1274/2577] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:01:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:02:08.432602 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=37 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:02:15.244464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c49741-0482-4301-870d-c5606dce47ad service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:02:15.246033 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-31c49741-0482-4301-870d-c5606dce47ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:02:15.276486 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c49741-0482-4301-870d-c5606dce47ad service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:02:15.280094 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7dd0b7-05c3-4d98-9d14-49793331655a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:02:15.280734 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7dd0b7-05c3-4d98-9d14-49793331655a service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:02:15.281015 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7dd0b7-05c3-4d98-9d14-49793331655a service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:02:15.281289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7dd0b7-05c3-4d98-9d14-49793331655a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:02:15.283172 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8d7dd0b7-05c3-4d98-9d14-49793331655a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:02:15.363363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7dd0b7-05c3-4d98-9d14-49793331655a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:02:15.364872 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7dd0b7-05c3-4d98-9d14-49793331655a service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:02:15.385931 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1304/2578] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:02:15 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 15:02:15.557678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aefac79-b130-4158-8e65-84c5209c6bdc service placement] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:02:15.566917 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aefac79-b130-4158-8e65-84c5209c6bdc service placement] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:02:15.570514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22257c-d363-400b-b3ae-209edcababe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:02:15.571193 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22257c-d363-400b-b3ae-209edcababe2 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:02:15.571479 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22257c-d363-400b-b3ae-209edcababe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:02:15.571758 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22257c-d363-400b-b3ae-209edcababe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:02:15.573544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0d22257c-d363-400b-b3ae-209edcababe2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:02:15.660674 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22257c-d363-400b-b3ae-209edcababe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:02:15.662202 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22257c-d363-400b-b3ae-209edcababe2 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:02:15.684309 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1275/2579] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:02:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:02:18.835418 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7dd0b7-05c3-4d98-9d14-49793331655a service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:02:18.847557 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7dd0b7-05c3-4d98-9d14-49793331655a service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:02:18.853691 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d6a00-c8b8-4d13-873b-c250d87def0a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c1c13515a8b41a781478dc65e6a88b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:02:18.854368 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d6a00-c8b8-4d13-873b-c250d87def0a service placement] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:02:18.854666 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d6a00-c8b8-4d13-873b-c250d87def0a service placement] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:02:18.854977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d6a00-c8b8-4d13-873b-c250d87def0a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:02:18.856811 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-906d6a00-c8b8-4d13-873b-c250d87def0a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:02:18.869464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d6a00-c8b8-4d13-873b-c250d87def0a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:02:18.871165 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d6a00-c8b8-4d13-873b-c250d87def0a service placement] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:02:18.882728 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1305/2580] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:02:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 15:02:18.898377 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22257c-d363-400b-b3ae-209edcababe2 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:02:18.899955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0d22257c-d363-400b-b3ae-209edcababe2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:02:18.918286 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22257c-d363-400b-b3ae-209edcababe2 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:02:18.924336 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc32800-5106-47cc-9fb6-9385d6b6b070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:02:18.925066 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc32800-5106-47cc-9fb6-9385d6b6b070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:02:18.925353 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc32800-5106-47cc-9fb6-9385d6b6b070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:02:18.925643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc32800-5106-47cc-9fb6-9385d6b6b070 admin admin] PATH_INFO: `/v3/users/32e6ace174a840f78c8d570719c3f631` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:02:18.927057 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc32800-5106-47cc-9fb6-9385d6b6b070 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32e6ace174a840f78c8d570719c3f631)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:02:18.929058 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc32800-5106-47cc-9fb6-9385d6b6b070 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:02:18.979069 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-abc32800-5106-47cc-9fb6-9385d6b6b070 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32e6ace174a840f78c8d570719c3f631'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:02:18.986141 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-abc32800-5106-47cc-9fb6-9385d6b6b070 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32e6ace174a840f78c8d570719c3f631'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:02:18.993356 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-abc32800-5106-47cc-9fb6-9385d6b6b070 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32e6ace174a840f78c8d570719c3f631'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:02:19.005000 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-abc32800-5106-47cc-9fb6-9385d6b6b070 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32e6ace174a840f78c8d570719c3f631'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:02:19.006009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-abc32800-5106-47cc-9fb6-9385d6b6b070 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32e6ace174a840f78c8d570719c3f631'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:02:19.011836 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1276/2581] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:02:18 2022] DELETE /identity/v3/users/32e6ace174a840f78c8d570719c3f631 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:02:19.184534 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d6a00-c8b8-4d13-873b-c250d87def0a service placement] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:02:19.196633 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d6a00-c8b8-4d13-873b-c250d87def0a service placement] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:02:19.253121 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f726b53-b506-49c2-a234-fb681f7645f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:02:19.254000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f726b53-b506-49c2-a234-fb681f7645f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:02:19.254288 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f726b53-b506-49c2-a234-fb681f7645f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:02:19.254571 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f726b53-b506-49c2-a234-fb681f7645f8 admin admin] PATH_INFO: `/v3/projects/502541429884445e8426837b3fd0d75e` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:02:19.255793 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f726b53-b506-49c2-a234-fb681f7645f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=502541429884445e8426837b3fd0d75e)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:02:19.258201 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f726b53-b506-49c2-a234-fb681f7645f8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:02:19.298973 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:02:19.307101 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0f726b53-b506-49c2-a234-fb681f7645f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '502541429884445e8426837b3fd0d75e', 'request_id': 'req-0f726b53-b506-49c2-a234-fb681f7645f8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:02:19.313063 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0f726b53-b506-49c2-a234-fb681f7645f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '502541429884445e8426837b3fd0d75e', 'request_id': 'req-0f726b53-b506-49c2-a234-fb681f7645f8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:02:19.321395 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0f726b53-b506-49c2-a234-fb681f7645f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '502541429884445e8426837b3fd0d75e', 'request_id': 'req-0f726b53-b506-49c2-a234-fb681f7645f8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:02:19.335251 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0f726b53-b506-49c2-a234-fb681f7645f8 admin admin] The token cache is being invalidate because project 502541429884445e8426837b3fd0d75e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:02:19.335547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0f726b53-b506-49c2-a234-fb681f7645f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:02:19.336264 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-0f726b53-b506-49c2-a234-fb681f7645f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:02:19.337918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1306/2582] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:02:19 2022] DELETE /identity/v3/projects/502541429884445e8426837b3fd0d75e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:02:19.344134 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f0ecd-a0c3-4738-af82-0850d5d7b14a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:02:19.344503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f0ecd-a0c3-4738-af82-0850d5d7b14a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:02:19.344704 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f0ecd-a0c3-4738-af82-0850d5d7b14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:02:19.417831 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:02:19.422233 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-fe7f0ecd-a0c3-4738-af82-0850d5d7b14a None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:02:19.424073 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fe7f0ecd-a0c3-4738-af82-0850d5d7b14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:02:19.489083 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1277/2583] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:02:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:02:19.497761 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad2341-afbd-4676-ab76-44a4fa780e2c None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:02:19.498054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad2341-afbd-4676-ab76-44a4fa780e2c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:02:19.498336 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad2341-afbd-4676-ab76-44a4fa780e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:02:19.518224 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-aaad2341-afbd-4676-ab76-44a4fa780e2c None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:02:19.519493 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-aaad2341-afbd-4676-ab76-44a4fa780e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:02:19.528058 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1307/2584] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:02:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:02:19.532619 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f0ecd-a0c3-4738-af82-0850d5d7b14a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:02:19.543482 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f0ecd-a0c3-4738-af82-0850d5d7b14a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:02:19.548640 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc1449-2c83-4f16-9088-82247ea89091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:02:19.549333 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc1449-2c83-4f16-9088-82247ea89091 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:02:19.549624 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc1449-2c83-4f16-9088-82247ea89091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:02:19.549909 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc1449-2c83-4f16-9088-82247ea89091 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:02:19.550680 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc1449-2c83-4f16-9088-82247ea89091 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 15:02:19.551001 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc1449-2c83-4f16-9088-82247ea89091 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:02:19.552627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc1449-2c83-4f16-9088-82247ea89091 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:02:19.566280 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1278/2585] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 15:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 15:02:19.570533 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad2341-afbd-4676-ab76-44a4fa780e2c None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:02:19.580226 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad2341-afbd-4676-ab76-44a4fa780e2c None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:02:19.585465 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608af77-48b9-43d4-9d56-fd3fd048edab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:02:19.586137 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608af77-48b9-43d4-9d56-fd3fd048edab admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:02:19.586420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608af77-48b9-43d4-9d56-fd3fd048edab admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:02:19.586724 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608af77-48b9-43d4-9d56-fd3fd048edab admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:02:19.587428 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608af77-48b9-43d4-9d56-fd3fd048edab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:02:19.589340 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608af77-48b9-43d4-9d56-fd3fd048edab admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:02:19.628017 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1308/2586] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 15:02:19 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:02:19.632770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc1449-2c83-4f16-9088-82247ea89091 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:02:19.643083 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc1449-2c83-4f16-9088-82247ea89091 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:02:19.704323 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65b85c-d106-49ce-989d-1197ca1e9e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:02:19.705077 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65b85c-d106-49ce-989d-1197ca1e9e05 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:02:19.705369 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65b85c-d106-49ce-989d-1197ca1e9e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:02:19.705652 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65b85c-d106-49ce-989d-1197ca1e9e05 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:02:19.706362 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac65b85c-d106-49ce-989d-1197ca1e9e05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:02:19.708351 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac65b85c-d106-49ce-989d-1197ca1e9e05 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:02:19.730715 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1279/2587] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 15:02:19 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 15:02:19.734912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608af77-48b9-43d4-9d56-fd3fd048edab admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:02:19.745388 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608af77-48b9-43d4-9d56-fd3fd048edab admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:02:19.750065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0648c71-b63e-4ec6-b30a-a182a75bb930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:02:19.750772 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0648c71-b63e-4ec6-b30a-a182a75bb930 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:02:19.751064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0648c71-b63e-4ec6-b30a-a182a75bb930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:02:19.751365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0648c71-b63e-4ec6-b30a-a182a75bb930 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:02:19.752082 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0648c71-b63e-4ec6-b30a-a182a75bb930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:02:19.753717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0648c71-b63e-4ec6-b30a-a182a75bb930 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:02:19.767310 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1309/2588] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:02:19.771413 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65b85c-d106-49ce-989d-1197ca1e9e05 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:02:19.782004 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65b85c-d106-49ce-989d-1197ca1e9e05 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:02:19.786703 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d672287-db38-414d-b109-ccc68a731394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:02:19.788046 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d672287-db38-414d-b109-ccc68a731394 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:02:19.788353 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d672287-db38-414d-b109-ccc68a731394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:02:19.788636 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d672287-db38-414d-b109-ccc68a731394 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:02:19.789384 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d672287-db38-414d-b109-ccc68a731394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:02:19.791009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d672287-db38-414d-b109-ccc68a731394 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:02:19.801341 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1280/2589] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:02:19.805548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0648c71-b63e-4ec6-b30a-a182a75bb930 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:02:19.815594 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0648c71-b63e-4ec6-b30a-a182a75bb930 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:02:19.820738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b25ec-7d67-42ec-9175-c1838e9db192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:02:19.821571 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b25ec-7d67-42ec-9175-c1838e9db192 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:02:19.821837 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b25ec-7d67-42ec-9175-c1838e9db192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:02:19.822115 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b25ec-7d67-42ec-9175-c1838e9db192 admin admin] PATH_INFO: `/v3/projects/5bd34785f6d847ae8cb5be4e07d29c38/users/7a8df9e916ba4542af81e2a29229c1db/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:02:19.838437 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b25ec-7d67-42ec-9175-c1838e9db192 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bd34785f6d847ae8cb5be4e07d29c38, user_id=7a8df9e916ba4542af81e2a29229c1db, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:02:19.841095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b25ec-7d67-42ec-9175-c1838e9db192 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:02:19.855008 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1310/2590] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:02:19 2022] PUT /identity/v3/projects/5bd34785f6d847ae8cb5be4e07d29c38/users/7a8df9e916ba4542af81e2a29229c1db/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:02:19.860202 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46816c1d-9625-4050-906e-5bf3dfee7644 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:02:19.860496 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46816c1d-9625-4050-906e-5bf3dfee7644 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:02:19.860781 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46816c1d-9625-4050-906e-5bf3dfee7644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:02:19.878724 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-46816c1d-9625-4050-906e-5bf3dfee7644 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:02:19.879951 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-46816c1d-9625-4050-906e-5bf3dfee7644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:02:19.938682 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1281/2591] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:02:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:02:19.949012 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b25ec-7d67-42ec-9175-c1838e9db192 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:02:19.950751 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6b3b25ec-7d67-42ec-9175-c1838e9db192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:02:20.008018 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b25ec-7d67-42ec-9175-c1838e9db192 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:02:20.011779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7179c52a-7b5c-4e92-b868-28b18ff9518f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:02:20.012464 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179c52a-7b5c-4e92-b868-28b18ff9518f service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:02:20.012752 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179c52a-7b5c-4e92-b868-28b18ff9518f service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:02:20.013034 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179c52a-7b5c-4e92-b868-28b18ff9518f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:02:20.023648 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7179c52a-7b5c-4e92-b868-28b18ff9518f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:02:20.025109 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7179c52a-7b5c-4e92-b868-28b18ff9518f service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:02:20.034988 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1311/2592] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:02:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:02:20.460664 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46816c1d-9625-4050-906e-5bf3dfee7644 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:02:20.462083 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-46816c1d-9625-4050-906e-5bf3dfee7644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:02:20.476400 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:02:20.542110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46816c1d-9625-4050-906e-5bf3dfee7644 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:02:20.548219 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58127d-3de8-4b7b-b42b-fe63a8edbee5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:02:20.548896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58127d-3de8-4b7b-b42b-fe63a8edbee5 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:02:20.549188 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58127d-3de8-4b7b-b42b-fe63a8edbee5 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:02:20.549461 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58127d-3de8-4b7b-b42b-fe63a8edbee5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:02:20.551328 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4a58127d-3de8-4b7b-b42b-fe63a8edbee5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:02:20.636460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a58127d-3de8-4b7b-b42b-fe63a8edbee5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:02:20.638095 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a58127d-3de8-4b7b-b42b-fe63a8edbee5 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:02:20.660518 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1282/2593] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:02:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:02:23.865382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065065d2-a610-4fa0-9547-30bf6f7d2e2a None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:02:23.865952 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065065d2-a610-4fa0-9547-30bf6f7d2e2a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:02:23.865952 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065065d2-a610-4fa0-9547-30bf6f7d2e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:02:23.892959 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-065065d2-a610-4fa0-9547-30bf6f7d2e2a None None] MFA Rules not processed for user `7a8df9e916ba4542af81e2a29229c1db`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:02:23.894234 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-065065d2-a610-4fa0-9547-30bf6f7d2e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:02:23.944701 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:02:23.969341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1312/2594] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:02:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:02:23.984513 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58127d-3de8-4b7b-b42b-fe63a8edbee5 service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:02:23.985772 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4a58127d-3de8-4b7b-b42b-fe63a8edbee5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:02:24.001100 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58127d-3de8-4b7b-b42b-fe63a8edbee5 service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:02:24.007255 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063bf5f1-8402-4e85-895d-367fa042d804 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:02:24.007924 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bf5f1-8402-4e85-895d-367fa042d804 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:02:24.008210 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bf5f1-8402-4e85-895d-367fa042d804 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:02:24.008488 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bf5f1-8402-4e85-895d-367fa042d804 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:02:24.017350 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063bf5f1-8402-4e85-895d-367fa042d804 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:02:24.018978 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063bf5f1-8402-4e85-895d-367fa042d804 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:02:24.029113 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1283/2595] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:02:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:02:25.721594 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9787b-0cba-4aa3-b315-f6a1b54664f9 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:02:25.722237 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9787b-0cba-4aa3-b315-f6a1b54664f9 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:02:25.722237 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9787b-0cba-4aa3-b315-f6a1b54664f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:02:25.740001 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-7ff9787b-0cba-4aa3-b315-f6a1b54664f9 None None] MFA Rules not processed for user `7a8df9e916ba4542af81e2a29229c1db`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:02:25.741245 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7ff9787b-0cba-4aa3-b315-f6a1b54664f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:02:25.747983 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1313/2596] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:02:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:02:26.059901 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063bf5f1-8402-4e85-895d-367fa042d804 service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:02:26.061483 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-063bf5f1-8402-4e85-895d-367fa042d804 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:02:26.074249 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063bf5f1-8402-4e85-895d-367fa042d804 service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:02:26.077908 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067483ca-0800-4471-8066-983e52403a1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:02:26.078560 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067483ca-0800-4471-8066-983e52403a1c service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:02:26.078887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067483ca-0800-4471-8066-983e52403a1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:02:26.079172 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067483ca-0800-4471-8066-983e52403a1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:02:26.080934 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-067483ca-0800-4471-8066-983e52403a1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:02:26.091310 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067483ca-0800-4471-8066-983e52403a1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:02:26.092749 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067483ca-0800-4471-8066-983e52403a1c service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:02:26.100905 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1284/2597] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:02:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:02:26.505120 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9787b-0cba-4aa3-b315-f6a1b54664f9 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:02:26.506599 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7ff9787b-0cba-4aa3-b315-f6a1b54664f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:02:26.608232 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9787b-0cba-4aa3-b315-f6a1b54664f9 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:02:26.613637 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def42daf-1a40-4ea3-b3f5-ec59e7059eb5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c1c13515a8b41a781478dc65e6a88b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:02:26.614311 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def42daf-1a40-4ea3-b3f5-ec59e7059eb5 service placement] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:02:26.614601 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def42daf-1a40-4ea3-b3f5-ec59e7059eb5 service placement] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:02:26.614912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def42daf-1a40-4ea3-b3f5-ec59e7059eb5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:02:26.616811 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-def42daf-1a40-4ea3-b3f5-ec59e7059eb5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:02:26.628956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def42daf-1a40-4ea3-b3f5-ec59e7059eb5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:02:26.630562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def42daf-1a40-4ea3-b3f5-ec59e7059eb5 service placement] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:02:26.641551 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1314/2598] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:02:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:02:34.938042 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067483ca-0800-4471-8066-983e52403a1c service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:02:34.951773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067483ca-0800-4471-8066-983e52403a1c service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:02:34.957338 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e766d-6bbc-4655-adfe-76e3567fcf2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:02:34.958098 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e766d-6bbc-4655-adfe-76e3567fcf2a service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:02:34.958397 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e766d-6bbc-4655-adfe-76e3567fcf2a service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:02:34.958706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e766d-6bbc-4655-adfe-76e3567fcf2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:02:34.960526 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-672e766d-6bbc-4655-adfe-76e3567fcf2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:02:34.973458 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e766d-6bbc-4655-adfe-76e3567fcf2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:02:34.975687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e766d-6bbc-4655-adfe-76e3567fcf2a service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:02:34.987420 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1285/2599] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:02:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:02:36.637003 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=41 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:02:38.293293 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def42daf-1a40-4ea3-b3f5-ec59e7059eb5 service placement] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:02:38.306181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def42daf-1a40-4ea3-b3f5-ec59e7059eb5 service placement] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:02:38.312008 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a641d-917c-4e00-9c3e-a0db4e531fed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:02:38.312741 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a641d-917c-4e00-9c3e-a0db4e531fed service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:02:38.312940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a641d-917c-4e00-9c3e-a0db4e531fed service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:02:38.313220 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a641d-917c-4e00-9c3e-a0db4e531fed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:02:38.315079 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-613a641d-917c-4e00-9c3e-a0db4e531fed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:02:38.328903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a641d-917c-4e00-9c3e-a0db4e531fed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:02:38.330517 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a641d-917c-4e00-9c3e-a0db4e531fed service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:02:38.343285 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1315/2600] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:02:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:02:44.982184 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=81 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:02:48.337758 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=6 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:04:15.591128 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e766d-6bbc-4655-adfe-76e3567fcf2a service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:04:15.592607 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-672e766d-6bbc-4655-adfe-76e3567fcf2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:04:15.660675 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e766d-6bbc-4655-adfe-76e3567fcf2a service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:04:15.664521 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01a4d2-66ea-4fee-a850-9c56cff6c94c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:04:15.665181 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01a4d2-66ea-4fee-a850-9c56cff6c94c service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:04:15.665464 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01a4d2-66ea-4fee-a850-9c56cff6c94c service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:04:15.665743 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01a4d2-66ea-4fee-a850-9c56cff6c94c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:04:15.667609 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1f01a4d2-66ea-4fee-a850-9c56cff6c94c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:04:15.730109 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01a4d2-66ea-4fee-a850-9c56cff6c94c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:04:15.731578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01a4d2-66ea-4fee-a850-9c56cff6c94c service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:04:15.750205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1286/2601] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:04:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:04:24.452546 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a641d-917c-4e00-9c3e-a0db4e531fed service nova] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:04:24.463972 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a641d-917c-4e00-9c3e-a0db4e531fed service nova] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:04:24.467690 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e7f6e-e35d-4f97-951b-e3d5ac912459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:04:24.468346 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e7f6e-e35d-4f97-951b-e3d5ac912459 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:04:24.468629 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e7f6e-e35d-4f97-951b-e3d5ac912459 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:04:24.468901 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e7f6e-e35d-4f97-951b-e3d5ac912459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:04:24.470813 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f42e7f6e-e35d-4f97-951b-e3d5ac912459 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:04:24.493425 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e7f6e-e35d-4f97-951b-e3d5ac912459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:04:24.496216 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e7f6e-e35d-4f97-951b-e3d5ac912459 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:04:24.526551 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1316/2602] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:04:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:04:25.628664 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01a4d2-66ea-4fee-a850-9c56cff6c94c service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:04:25.638504 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01a4d2-66ea-4fee-a850-9c56cff6c94c service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:04:25.642210 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13dfd49-c9bd-4563-97c5-1e09358933fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:04:25.643442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13dfd49-c9bd-4563-97c5-1e09358933fb service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:04:25.643753 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13dfd49-c9bd-4563-97c5-1e09358933fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:04:25.644034 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13dfd49-c9bd-4563-97c5-1e09358933fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:04:25.645938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d13dfd49-c9bd-4563-97c5-1e09358933fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:04:25.660305 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13dfd49-c9bd-4563-97c5-1e09358933fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:04:25.661734 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13dfd49-c9bd-4563-97c5-1e09358933fb service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:04:25.673042 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1287/2603] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:04:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:04:34.523768 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=13 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:04:35.667896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=62 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:04:59.070991 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e7f6e-e35d-4f97-951b-e3d5ac912459 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:04:59.090211 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e7f6e-e35d-4f97-951b-e3d5ac912459 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:04:59.094319 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff61a6-bcae-40c1-aea6-fd18bd971867 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:04:59.095573 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff61a6-bcae-40c1-aea6-fd18bd971867 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:04:59.095796 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff61a6-bcae-40c1-aea6-fd18bd971867 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:04:59.096072 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff61a6-bcae-40c1-aea6-fd18bd971867 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:04:59.097902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-eaff61a6-bcae-40c1-aea6-fd18bd971867 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:04:59.108234 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff61a6-bcae-40c1-aea6-fd18bd971867 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:04:59.109736 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff61a6-bcae-40c1-aea6-fd18bd971867 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:04:59.118969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1317/2604] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:04:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:05:09.115117 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=10 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:05:25.348826 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13dfd49-c9bd-4563-97c5-1e09358933fb service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:05:25.350433 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d13dfd49-c9bd-4563-97c5-1e09358933fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:05:25.366907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13dfd49-c9bd-4563-97c5-1e09358933fb service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:05:25.376064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553c781-f1cb-478b-9a26-b13f7140dd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:05:25.377899 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553c781-f1cb-478b-9a26-b13f7140dd29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:25.378398 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553c781-f1cb-478b-9a26-b13f7140dd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:25.378925 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553c781-f1cb-478b-9a26-b13f7140dd29 admin admin] PATH_INFO: `/v3/users/3375ea629b5f4c20a0331a610a1712e1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:25.381211 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553c781-f1cb-478b-9a26-b13f7140dd29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3375ea629b5f4c20a0331a610a1712e1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:05:25.385016 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553c781-f1cb-478b-9a26-b13f7140dd29 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:05:25.463390 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8553c781-f1cb-478b-9a26-b13f7140dd29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3375ea629b5f4c20a0331a610a1712e1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:05:25.472965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8553c781-f1cb-478b-9a26-b13f7140dd29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3375ea629b5f4c20a0331a610a1712e1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:05:25.482317 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8553c781-f1cb-478b-9a26-b13f7140dd29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3375ea629b5f4c20a0331a610a1712e1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:05:25.496999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8553c781-f1cb-478b-9a26-b13f7140dd29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3375ea629b5f4c20a0331a610a1712e1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:05:25.497744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8553c781-f1cb-478b-9a26-b13f7140dd29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3375ea629b5f4c20a0331a610a1712e1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:05:25.504597 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1288/2605] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:05:25 2022] DELETE /identity/v3/users/3375ea629b5f4c20a0331a610a1712e1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:05:25.509440 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22293c2-8c7a-4b47-b945-2be8f85cd26e None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:25.510066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22293c2-8c7a-4b47-b945-2be8f85cd26e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:25.510066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22293c2-8c7a-4b47-b945-2be8f85cd26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:25.529502 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-d22293c2-8c7a-4b47-b945-2be8f85cd26e None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:05:25.530800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d22293c2-8c7a-4b47-b945-2be8f85cd26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:05:25.597283 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1318/2606] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:05:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:05:25.609236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553c781-f1cb-478b-9a26-b13f7140dd29 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:05:25.611560 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8553c781-f1cb-478b-9a26-b13f7140dd29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:05:25.712632 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553c781-f1cb-478b-9a26-b13f7140dd29 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:05:25.730690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d8165-1fd0-4ed3-8052-fa8943605c84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:05:25.731699 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d8165-1fd0-4ed3-8052-fa8943605c84 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:25.732134 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d8165-1fd0-4ed3-8052-fa8943605c84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:25.732554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d8165-1fd0-4ed3-8052-fa8943605c84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:25.748139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d8165-1fd0-4ed3-8052-fa8943605c84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:05:25.750732 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d8165-1fd0-4ed3-8052-fa8943605c84 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:05:25.767020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1289/2607] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:05:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:05:26.079429 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22293c2-8c7a-4b47-b945-2be8f85cd26e None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:05:26.081238 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d22293c2-8c7a-4b47-b945-2be8f85cd26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:05:26.098622 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22293c2-8c7a-4b47-b945-2be8f85cd26e None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:05:26.104045 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63413aca-f9f9-403d-a182-94bd915e95e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:05:26.105041 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63413aca-f9f9-403d-a182-94bd915e95e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:26.105386 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63413aca-f9f9-403d-a182-94bd915e95e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:26.105672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63413aca-f9f9-403d-a182-94bd915e95e5 admin admin] PATH_INFO: `/v3/projects/243bbd81047e44acb2e0110e4002eb00` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:26.107279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63413aca-f9f9-403d-a182-94bd915e95e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=243bbd81047e44acb2e0110e4002eb00)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:05:26.109838 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63413aca-f9f9-403d-a182-94bd915e95e5 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:05:26.184164 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-63413aca-f9f9-403d-a182-94bd915e95e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '243bbd81047e44acb2e0110e4002eb00', 'request_id': 'req-63413aca-f9f9-403d-a182-94bd915e95e5'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:05:26.191739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-63413aca-f9f9-403d-a182-94bd915e95e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '243bbd81047e44acb2e0110e4002eb00', 'request_id': 'req-63413aca-f9f9-403d-a182-94bd915e95e5'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:05:26.198670 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-63413aca-f9f9-403d-a182-94bd915e95e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '243bbd81047e44acb2e0110e4002eb00', 'request_id': 'req-63413aca-f9f9-403d-a182-94bd915e95e5'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:05:26.214312 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-63413aca-f9f9-403d-a182-94bd915e95e5 admin admin] The token cache is being invalidate because project 243bbd81047e44acb2e0110e4002eb00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:05:26.214926 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-63413aca-f9f9-403d-a182-94bd915e95e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:05:26.215638 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-63413aca-f9f9-403d-a182-94bd915e95e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:05:26.217352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1319/2608] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:05:26 2022] DELETE /identity/v3/projects/243bbd81047e44acb2e0110e4002eb00 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:05:26.224606 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc1751-f61d-4963-8d6a-6b195f50ddcf None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:26.224955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc1751-f61d-4963-8d6a-6b195f50ddcf None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:26.225246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc1751-f61d-4963-8d6a-6b195f50ddcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:26.246954 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-12dc1751-f61d-4963-8d6a-6b195f50ddcf None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:05:26.248102 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-12dc1751-f61d-4963-8d6a-6b195f50ddcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:05:26.342904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1290/2609] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:05:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 15:05:26.350954 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b23258-b469-49ed-9a0e-059df0195514 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:26.351393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b23258-b469-49ed-9a0e-059df0195514 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:26.351791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b23258-b469-49ed-9a0e-059df0195514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:26.371756 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a1b23258-b469-49ed-9a0e-059df0195514 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:05:26.373004 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a1b23258-b469-49ed-9a0e-059df0195514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:05:26.382319 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1320/2610] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:05:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 15:05:26.387854 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc1751-f61d-4963-8d6a-6b195f50ddcf None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:05:26.399403 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc1751-f61d-4963-8d6a-6b195f50ddcf None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:05:26.404309 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d8310-1cee-4925-b576-b0cf22f8c7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:05:26.405005 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d8310-1cee-4925-b576-b0cf22f8c7fa admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:26.405296 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d8310-1cee-4925-b576-b0cf22f8c7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:26.405579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d8310-1cee-4925-b576-b0cf22f8c7fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:26.406386 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d8310-1cee-4925-b576-b0cf22f8c7fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 15:05:26.406747 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d8310-1cee-4925-b576-b0cf22f8c7fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:05:26.408372 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d8310-1cee-4925-b576-b0cf22f8c7fa admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:05:26.424623 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1291/2611] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 15:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 15:05:26.429368 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b23258-b469-49ed-9a0e-059df0195514 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:05:26.439931 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b23258-b469-49ed-9a0e-059df0195514 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:05:26.445364 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e45306-73fe-44be-8621-3295e65945ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:05:26.446104 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e45306-73fe-44be-8621-3295e65945ef admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:26.446423 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e45306-73fe-44be-8621-3295e65945ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:26.446792 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e45306-73fe-44be-8621-3295e65945ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:26.447789 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e45306-73fe-44be-8621-3295e65945ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:05:26.449972 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e45306-73fe-44be-8621-3295e65945ef admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:05:26.479179 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1321/2612] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 15:05:26 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:05:26.483971 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d8310-1cee-4925-b576-b0cf22f8c7fa admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:05:26.490229 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:05:26.496802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d8310-1cee-4925-b576-b0cf22f8c7fa admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:05:26.572834 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538433a-f86a-4fa7-9399-69b6326a58b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:05:26.573548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538433a-f86a-4fa7-9399-69b6326a58b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:26.573837 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538433a-f86a-4fa7-9399-69b6326a58b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:26.574123 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538433a-f86a-4fa7-9399-69b6326a58b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:26.574902 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538433a-f86a-4fa7-9399-69b6326a58b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:05:26.576920 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538433a-f86a-4fa7-9399-69b6326a58b8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:05:26.601541 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1292/2613] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 15:05:26 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 15:05:26.606195 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e45306-73fe-44be-8621-3295e65945ef admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:05:26.620140 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e45306-73fe-44be-8621-3295e65945ef admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:05:26.625189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9ea29-e097-48c0-9ceb-a5cdbdac6e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:05:26.625885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9ea29-e097-48c0-9ceb-a5cdbdac6e40 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:26.626177 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9ea29-e097-48c0-9ceb-a5cdbdac6e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:26.626455 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9ea29-e097-48c0-9ceb-a5cdbdac6e40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:26.627269 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e9ea29-e097-48c0-9ceb-a5cdbdac6e40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:05:26.628921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e9ea29-e097-48c0-9ceb-a5cdbdac6e40 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:05:26.641452 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1322/2614] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 15:05:26.645968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538433a-f86a-4fa7-9399-69b6326a58b8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:05:26.659819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538433a-f86a-4fa7-9399-69b6326a58b8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:05:26.667551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49364893-983f-4d8a-8516-a76588630e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:05:26.668917 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49364893-983f-4d8a-8516-a76588630e1d admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:26.669412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49364893-983f-4d8a-8516-a76588630e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:26.669887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49364893-983f-4d8a-8516-a76588630e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:26.671224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49364893-983f-4d8a-8516-a76588630e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:05:26.675649 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49364893-983f-4d8a-8516-a76588630e1d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:05:26.688233 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1293/2615] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:05:26.695145 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9ea29-e097-48c0-9ceb-a5cdbdac6e40 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:05:26.713549 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9ea29-e097-48c0-9ceb-a5cdbdac6e40 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:05:26.722244 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5648705-595a-40fa-a065-1a29884ef6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:05:26.723911 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5648705-595a-40fa-a065-1a29884ef6b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:26.724433 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5648705-595a-40fa-a065-1a29884ef6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:26.724926 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5648705-595a-40fa-a065-1a29884ef6b8 admin admin] PATH_INFO: `/v3/projects/f24fdc4bc9ca442e948484e73713779c/users/180eebbd503e441b9332720e70f7f339/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:26.748339 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5648705-595a-40fa-a065-1a29884ef6b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f24fdc4bc9ca442e948484e73713779c, user_id=180eebbd503e441b9332720e70f7f339, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:05:26.751064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5648705-595a-40fa-a065-1a29884ef6b8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:05:26.765462 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1323/2616] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:05:26 2022] PUT /identity/v3/projects/f24fdc4bc9ca442e948484e73713779c/users/180eebbd503e441b9332720e70f7f339/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:05:26.771075 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b592b-a6d1-46cb-aec4-29d326a19249 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:26.771357 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b592b-a6d1-46cb-aec4-29d326a19249 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:26.771737 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b592b-a6d1-46cb-aec4-29d326a19249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:26.791460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-120b592b-a6d1-46cb-aec4-29d326a19249 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:05:26.792707 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-120b592b-a6d1-46cb-aec4-29d326a19249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:05:26.858548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1294/2617] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:05:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:05:26.870950 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5648705-595a-40fa-a065-1a29884ef6b8 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:05:26.872937 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b5648705-595a-40fa-a065-1a29884ef6b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:05:26.897829 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:05:26.942754 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5648705-595a-40fa-a065-1a29884ef6b8 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:05:26.946766 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cab8fb-01ff-44d1-a450-f47a7e38869c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:05:26.947435 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cab8fb-01ff-44d1-a450-f47a7e38869c service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:26.947723 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cab8fb-01ff-44d1-a450-f47a7e38869c service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:26.948027 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cab8fb-01ff-44d1-a450-f47a7e38869c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:26.958787 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cab8fb-01ff-44d1-a450-f47a7e38869c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:05:26.960316 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cab8fb-01ff-44d1-a450-f47a7e38869c service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:05:26.971572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1324/2618] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:05:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:05:31.977505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c0e80-de30-49b8-a893-b1ef622803a2 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:31.978095 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c0e80-de30-49b8-a893-b1ef622803a2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:31.978560 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c0e80-de30-49b8-a893-b1ef622803a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:32.024405 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-589c0e80-de30-49b8-a893-b1ef622803a2 None None] MFA Rules not processed for user `180eebbd503e441b9332720e70f7f339`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:05:32.026098 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-589c0e80-de30-49b8-a893-b1ef622803a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:05:32.100015 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1295/2619] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:05:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:05:32.104757 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cab8fb-01ff-44d1-a450-f47a7e38869c service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:05:32.116034 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cab8fb-01ff-44d1-a450-f47a7e38869c service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:05:32.129719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f3399-6168-4e21-9462-3862bcbf7d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:05:32.130852 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f3399-6168-4e21-9462-3862bcbf7d8d admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:32.131386 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f3399-6168-4e21-9462-3862bcbf7d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:32.131915 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f3399-6168-4e21-9462-3862bcbf7d8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:32.133318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f3399-6168-4e21-9462-3862bcbf7d8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:05:32.137365 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f3399-6168-4e21-9462-3862bcbf7d8d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:05:32.189998 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1325/2620] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 15:05:32 2022] POST /identity/v3/projects => generated 383 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:05:32.194440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c0e80-de30-49b8-a893-b1ef622803a2 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:05:32.206634 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c0e80-de30-49b8-a893-b1ef622803a2 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:05:32.217951 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:05:32.275329 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a790d-f46c-424f-b67a-10fb977d9b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:05:32.276088 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a790d-f46c-424f-b67a-10fb977d9b74 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:32.276644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a790d-f46c-424f-b67a-10fb977d9b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:32.276644 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a790d-f46c-424f-b67a-10fb977d9b74 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:32.277380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01a790d-f46c-424f-b67a-10fb977d9b74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:05:32.279374 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01a790d-f46c-424f-b67a-10fb977d9b74 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:05:32.301780 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1296/2621] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 15:05:32 2022] POST /identity/v3/users => generated 298 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:05:32.306626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f3399-6168-4e21-9462-3862bcbf7d8d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:05:32.322545 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f3399-6168-4e21-9462-3862bcbf7d8d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:05:32.331306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca445df3-e572-4e6a-ba59-8512f24cbae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:05:32.332787 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca445df3-e572-4e6a-ba59-8512f24cbae2 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:32.333374 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca445df3-e572-4e6a-ba59-8512f24cbae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:32.333924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca445df3-e572-4e6a-ba59-8512f24cbae2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:32.335106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca445df3-e572-4e6a-ba59-8512f24cbae2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:05:32.337558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca445df3-e572-4e6a-ba59-8512f24cbae2 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:05:32.354169 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1326/2622] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:05:32.358442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a790d-f46c-424f-b67a-10fb977d9b74 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:05:32.371026 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a790d-f46c-424f-b67a-10fb977d9b74 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:05:32.377114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d04db-5eb6-4159-99cc-a539352df990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:05:32.378066 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d04db-5eb6-4159-99cc-a539352df990 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:32.378987 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d04db-5eb6-4159-99cc-a539352df990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:32.384847 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d04db-5eb6-4159-99cc-a539352df990 admin admin] PATH_INFO: `/v3/projects/6b2fe1e09b7e4b43a978a345e7566a3a/users/8a00d14831a34f5ab9e455a327b01491/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:32.435934 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d04db-5eb6-4159-99cc-a539352df990 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b2fe1e09b7e4b43a978a345e7566a3a, user_id=8a00d14831a34f5ab9e455a327b01491, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:05:32.438932 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d04db-5eb6-4159-99cc-a539352df990 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:05:32.473749 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1297/2623] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:05:32 2022] PUT /identity/v3/projects/6b2fe1e09b7e4b43a978a345e7566a3a/users/8a00d14831a34f5ab9e455a327b01491/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:05:32.478995 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca445df3-e572-4e6a-ba59-8512f24cbae2 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:05:32.499701 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca445df3-e572-4e6a-ba59-8512f24cbae2 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:05:32.607013 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a448a9-4c53-4ad2-9c93-ac8a8a36415e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:05:32.607563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a448a9-4c53-4ad2-9c93-ac8a8a36415e admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:32.607563 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a448a9-4c53-4ad2-9c93-ac8a8a36415e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:32.608287 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a448a9-4c53-4ad2-9c93-ac8a8a36415e admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:32.609235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a448a9-4c53-4ad2-9c93-ac8a8a36415e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:05:32.611073 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a448a9-4c53-4ad2-9c93-ac8a8a36415e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:05:32.628857 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1327/2624] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:05:32.630056 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d04db-5eb6-4159-99cc-a539352df990 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:05:32.643698 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d04db-5eb6-4159-99cc-a539352df990 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:05:32.651912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fbc149-c7c5-4ece-84a5-ca989b466a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:05:32.653694 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fbc149-c7c5-4ece-84a5-ca989b466a02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:32.654454 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fbc149-c7c5-4ece-84a5-ca989b466a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:32.654942 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fbc149-c7c5-4ece-84a5-ca989b466a02 admin admin] PATH_INFO: `/v3/projects/6b2fe1e09b7e4b43a978a345e7566a3a/users/8a00d14831a34f5ab9e455a327b01491/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:32.658386 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fbc149-c7c5-4ece-84a5-ca989b466a02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b2fe1e09b7e4b43a978a345e7566a3a, user_id=8a00d14831a34f5ab9e455a327b01491, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:05:32.663290 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fbc149-c7c5-4ece-84a5-ca989b466a02 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:05:32.682302 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1298/2625] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:05:32 2022] PUT /identity/v3/projects/6b2fe1e09b7e4b43a978a345e7566a3a/users/8a00d14831a34f5ab9e455a327b01491/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 15:05:37.905160 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc350f50-73d7-49f4-b4dc-8c9c465985c0 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:37.906077 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc350f50-73d7-49f4-b4dc-8c9c465985c0 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:37.906077 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc350f50-73d7-49f4-b4dc-8c9c465985c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:37.936546 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-bc350f50-73d7-49f4-b4dc-8c9c465985c0 None None] MFA Rules not processed for user `8a00d14831a34f5ab9e455a327b01491`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:05:37.938255 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-bc350f50-73d7-49f4-b4dc-8c9c465985c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:05:38.036065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1328/2626] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:05:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:05:38.053510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fbc149-c7c5-4ece-84a5-ca989b466a02 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:05:38.055953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a3fbc149-c7c5-4ece-84a5-ca989b466a02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:05:38.161788 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fbc149-c7c5-4ece-84a5-ca989b466a02 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:05:38.166097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca95e25-9ca6-4b32-8482-682199813bb5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8a4b0dd4e21414d91fe7bed478d15e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:05:38.166806 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca95e25-9ca6-4b32-8482-682199813bb5 service glance] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:38.167098 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca95e25-9ca6-4b32-8482-682199813bb5 service glance] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:38.167415 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca95e25-9ca6-4b32-8482-682199813bb5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:38.232952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca95e25-9ca6-4b32-8482-682199813bb5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:05:38.234984 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca95e25-9ca6-4b32-8482-682199813bb5 service glance] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:05:38.239064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:05:38.248856 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1299/2627] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:05:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:05:38.422230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb020776-9e1c-4f52-a630-b716f404401e None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:38.422781 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb020776-9e1c-4f52-a630-b716f404401e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:38.423509 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb020776-9e1c-4f52-a630-b716f404401e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:38.456759 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:05:38.522554 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-bb020776-9e1c-4f52-a630-b716f404401e None None] MFA Rules not processed for user `0b7d100f2407432381d119c25814fff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:05:38.524222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-bb020776-9e1c-4f52-a630-b716f404401e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:05:38.658387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1329/2628] 10.222.0.67 () {58 vars in 1009 bytes} [Tue May 31 15:05:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:05:38.720843 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca95e25-9ca6-4b32-8482-682199813bb5 service glance] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:05:38.722227 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bca95e25-9ca6-4b32-8482-682199813bb5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:05:38.843853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca95e25-9ca6-4b32-8482-682199813bb5 service glance] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:05:38.851875 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782112d-6878-4fad-9046-780e12287071 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfa2282f957941b98db1d7497da3351b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:05:38.853052 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782112d-6878-4fad-9046-780e12287071 service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:38.853636 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782112d-6878-4fad-9046-780e12287071 service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:38.854129 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782112d-6878-4fad-9046-780e12287071 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:38.865423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782112d-6878-4fad-9046-780e12287071 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:05:38.868718 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782112d-6878-4fad-9046-780e12287071 service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:05:38.882746 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1300/2629] 10.222.0.67 () {60 vars in 1384 bytes} [Tue May 31 15:05:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 31 15:05:48.256967 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2a055-f17d-422d-9007-894c6b065803 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:48.257574 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2a055-f17d-422d-9007-894c6b065803 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:48.257689 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2a055-f17d-422d-9007-894c6b065803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:48.276217 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-83d2a055-f17d-422d-9007-894c6b065803 None None] MFA Rules not processed for user `180eebbd503e441b9332720e70f7f339`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:05:48.277446 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-83d2a055-f17d-422d-9007-894c6b065803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:05:48.285211 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1330/2630] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:05:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:05:48.876206 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=42 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:05:50.365145 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7211ed-adaa-4aef-8f56-49313067a469 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:50.366114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7211ed-adaa-4aef-8f56-49313067a469 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:50.366114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7211ed-adaa-4aef-8f56-49313067a469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:50.387039 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5a7211ed-adaa-4aef-8f56-49313067a469 None None] MFA Rules not processed for user `0b7d100f2407432381d119c25814fff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:05:50.388336 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5a7211ed-adaa-4aef-8f56-49313067a469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:05:50.396249 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1301/2631] 10.222.0.67 () {58 vars in 1009 bytes} [Tue May 31 15:05:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:05:50.411867 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2a055-f17d-422d-9007-894c6b065803 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:05:50.426811 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2a055-f17d-422d-9007-894c6b065803 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:05:50.433236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076b795-92ff-4a64-a7bc-9e340fc9a783 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfa2282f957941b98db1d7497da3351b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:05:50.434270 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076b795-92ff-4a64-a7bc-9e340fc9a783 service swift] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:05:50.434701 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076b795-92ff-4a64-a7bc-9e340fc9a783 service swift] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:05:50.435106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076b795-92ff-4a64-a7bc-9e340fc9a783 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:05:50.447934 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076b795-92ff-4a64-a7bc-9e340fc9a783 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:05:50.449577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076b795-92ff-4a64-a7bc-9e340fc9a783 service swift] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:05:50.459411 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1331/2632] 10.222.0.67 () {60 vars in 1384 bytes} [Tue May 31 15:05:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 31 15:06:00.380386 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:06:00.455448 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=48 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:06:03.818158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7211ed-adaa-4aef-8f56-49313067a469 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:06:03.819780 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5a7211ed-adaa-4aef-8f56-49313067a469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:06:03.957951 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7211ed-adaa-4aef-8f56-49313067a469 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:06:03.964403 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaeb759-6cc3-4175-ae32-eb4b6791a165 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:06:03.965093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaeb759-6cc3-4175-ae32-eb4b6791a165 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:06:03.965478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaeb759-6cc3-4175-ae32-eb4b6791a165 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:06:03.965766 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaeb759-6cc3-4175-ae32-eb4b6791a165 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:06:03.968952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7eaeb759-6cc3-4175-ae32-eb4b6791a165 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:06:03.994293 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaeb759-6cc3-4175-ae32-eb4b6791a165 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:06:03.997346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaeb759-6cc3-4175-ae32-eb4b6791a165 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:06:04.017483 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1302/2633] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:06:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 15:06:14.008940 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=34 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:07:04.344860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076b795-92ff-4a64-a7bc-9e340fc9a783 service swift] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:07:04.346373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d076b795-92ff-4a64-a7bc-9e340fc9a783 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:07:04.426712 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076b795-92ff-4a64-a7bc-9e340fc9a783 service swift] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:07:04.432150 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea5096-b6d9-4dd8-8424-4df8a0c9f36d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c1c13515a8b41a781478dc65e6a88b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:07:04.432848 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea5096-b6d9-4dd8-8424-4df8a0c9f36d service placement] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:07:04.433135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea5096-b6d9-4dd8-8424-4df8a0c9f36d service placement] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:07:04.433412 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea5096-b6d9-4dd8-8424-4df8a0c9f36d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:07:04.435465 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-97ea5096-b6d9-4dd8-8424-4df8a0c9f36d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:07:04.450471 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea5096-b6d9-4dd8-8424-4df8a0c9f36d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:07:04.452849 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea5096-b6d9-4dd8-8424-4df8a0c9f36d service placement] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:07:04.464128 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1332/2634] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:07:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:07:14.459318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=32 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:07:25.539456 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaeb759-6cc3-4175-ae32-eb4b6791a165 service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:07:25.540891 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7eaeb759-6cc3-4175-ae32-eb4b6791a165 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:07:25.608582 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaeb759-6cc3-4175-ae32-eb4b6791a165 service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:07:25.612169 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550a804-f30e-4c37-82c4-d5575615f2bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:07:25.612815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550a804-f30e-4c37-82c4-d5575615f2bc service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:07:25.613093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550a804-f30e-4c37-82c4-d5575615f2bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:07:25.613364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550a804-f30e-4c37-82c4-d5575615f2bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:07:25.615280 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7550a804-f30e-4c37-82c4-d5575615f2bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:07:25.672668 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550a804-f30e-4c37-82c4-d5575615f2bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:07:25.674100 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550a804-f30e-4c37-82c4-d5575615f2bc service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:07:25.682926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1303/2635] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:07:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:07:25.731182 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea5096-b6d9-4dd8-8424-4df8a0c9f36d service placement] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:07:25.741094 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea5096-b6d9-4dd8-8424-4df8a0c9f36d service placement] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:07:25.744692 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d2269-6c87-4998-b1bd-3308e6b2491c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:07:25.745334 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d2269-6c87-4998-b1bd-3308e6b2491c service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:07:25.745625 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d2269-6c87-4998-b1bd-3308e6b2491c service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:07:25.745902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d2269-6c87-4998-b1bd-3308e6b2491c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:07:25.748387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5b5d2269-6c87-4998-b1bd-3308e6b2491c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:07:25.807978 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d2269-6c87-4998-b1bd-3308e6b2491c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:07:25.809467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d2269-6c87-4998-b1bd-3308e6b2491c service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:07:25.819408 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1333/2636] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:07:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:07:35.678949 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=53 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:07:35.815134 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=28 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:09:14.587222 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550a804-f30e-4c37-82c4-d5575615f2bc service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:09:14.599804 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550a804-f30e-4c37-82c4-d5575615f2bc service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:09:14.605774 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbeb48-86a3-4c15-a7bc-d0ca4a4c8387 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:09:14.606428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbeb48-86a3-4c15-a7bc-d0ca4a4c8387 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:09:14.606748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbeb48-86a3-4c15-a7bc-d0ca4a4c8387 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:09:14.607042 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbeb48-86a3-4c15-a7bc-d0ca4a4c8387 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:09:14.608832 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e0cbeb48-86a3-4c15-a7bc-d0ca4a4c8387 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:09:14.702104 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cbeb48-86a3-4c15-a7bc-d0ca4a4c8387 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:09:14.704306 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cbeb48-86a3-4c15-a7bc-d0ca4a4c8387 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:09:14.714832 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1304/2637] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:09:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:09:24.710080 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=37 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:10:03.098439 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d2269-6c87-4998-b1bd-3308e6b2491c service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:10:03.153755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d2269-6c87-4998-b1bd-3308e6b2491c service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:10:03.157674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f7b44-a8e5-41d2-9f5c-ac365967273b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:10:03.158331 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f7b44-a8e5-41d2-9f5c-ac365967273b service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:10:03.158617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f7b44-a8e5-41d2-9f5c-ac365967273b service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:10:03.158962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f7b44-a8e5-41d2-9f5c-ac365967273b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:10:03.161670 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8e3f7b44-a8e5-41d2-9f5c-ac365967273b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:10:03.189134 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f7b44-a8e5-41d2-9f5c-ac365967273b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:10:03.190590 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f7b44-a8e5-41d2-9f5c-ac365967273b service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:10:03.199523 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1334/2638] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:10:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:10:13.195086 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=31 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:10:38.184937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbeb48-86a3-4c15-a7bc-d0ca4a4c8387 service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:10:38.194518 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbeb48-86a3-4c15-a7bc-d0ca4a4c8387 service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:10:38.198769 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb49393-3f3b-4844-b30a-4159500caff9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:10:38.199418 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb49393-3f3b-4844-b30a-4159500caff9 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:10:38.199699 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb49393-3f3b-4844-b30a-4159500caff9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:10:38.199976 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb49393-3f3b-4844-b30a-4159500caff9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:10:38.208481 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb49393-3f3b-4844-b30a-4159500caff9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:10:38.209900 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb49393-3f3b-4844-b30a-4159500caff9 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:10:38.218815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1305/2639] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:10:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:10:48.214793 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=6 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:11:25.977351 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f7b44-a8e5-41d2-9f5c-ac365967273b service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:11:25.987009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-8e3f7b44-a8e5-41d2-9f5c-ac365967273b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:11:25.990367 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f7b44-a8e5-41d2-9f5c-ac365967273b service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:11:25.995187 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59fc963-f5d7-4833-87c0-cfba7f7d52a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:11:25.996095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59fc963-f5d7-4833-87c0-cfba7f7d52a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:11:25.997166 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59fc963-f5d7-4833-87c0-cfba7f7d52a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:11:25.997517 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59fc963-f5d7-4833-87c0-cfba7f7d52a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:11:26.000203 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e59fc963-f5d7-4833-87c0-cfba7f7d52a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:11:26.075824 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59fc963-f5d7-4833-87c0-cfba7f7d52a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:11:26.077157 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb49393-3f3b-4844-b30a-4159500caff9 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:11:26.077648 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59fc963-f5d7-4833-87c0-cfba7f7d52a5 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:11:26.087652 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb49393-3f3b-4844-b30a-4159500caff9 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:11:26.088625 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1335/2640] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:11:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:11:26.091290 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe726d-5910-45dc-aa7f-662bb6ff2b2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:11:26.091951 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe726d-5910-45dc-aa7f-662bb6ff2b2d service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:11:26.092228 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe726d-5910-45dc-aa7f-662bb6ff2b2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:11:26.092500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe726d-5910-45dc-aa7f-662bb6ff2b2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:11:26.094386 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f9fe726d-5910-45dc-aa7f-662bb6ff2b2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:11:26.121792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe726d-5910-45dc-aa7f-662bb6ff2b2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:11:26.123430 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe726d-5910-45dc-aa7f-662bb6ff2b2d service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:11:26.136001 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1306/2641] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:11:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:11:27.747866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59fc963-f5d7-4833-87c0-cfba7f7d52a5 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:11:27.769820 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59fc963-f5d7-4833-87c0-cfba7f7d52a5 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:11:27.780839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aec805-a2c7-4d9d-8083-52fee9e0eaf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:11:27.781573 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aec805-a2c7-4d9d-8083-52fee9e0eaf6 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:11:27.781957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aec805-a2c7-4d9d-8083-52fee9e0eaf6 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:11:27.782240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aec805-a2c7-4d9d-8083-52fee9e0eaf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:11:27.801122 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aec805-a2c7-4d9d-8083-52fee9e0eaf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:11:27.802880 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aec805-a2c7-4d9d-8083-52fee9e0eaf6 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:11:27.818994 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1336/2642] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:11:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:11:29.812735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe726d-5910-45dc-aa7f-662bb6ff2b2d service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:11:29.814989 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f9fe726d-5910-45dc-aa7f-662bb6ff2b2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:11:29.837619 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe726d-5910-45dc-aa7f-662bb6ff2b2d service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:11:29.847331 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03b77b-be43-45bb-8f2e-0968dfc5d370 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c1c13515a8b41a781478dc65e6a88b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:11:29.848994 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03b77b-be43-45bb-8f2e-0968dfc5d370 service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:11:29.849558 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03b77b-be43-45bb-8f2e-0968dfc5d370 service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:11:29.850084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03b77b-be43-45bb-8f2e-0968dfc5d370 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:11:29.853528 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7c03b77b-be43-45bb-8f2e-0968dfc5d370 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:11:29.879829 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c03b77b-be43-45bb-8f2e-0968dfc5d370 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:11:29.882495 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c03b77b-be43-45bb-8f2e-0968dfc5d370 service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:11:29.912403 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1307/2643] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:11:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:11:31.255989 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae67a0-1771-4442-8fd2-c3617bff23f8 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:11:31.256544 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae67a0-1771-4442-8fd2-c3617bff23f8 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:11:31.256544 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae67a0-1771-4442-8fd2-c3617bff23f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:11:31.273231 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-94ae67a0-1771-4442-8fd2-c3617bff23f8 None None] MFA Rules not processed for user `7a8df9e916ba4542af81e2a29229c1db`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:11:31.274459 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-94ae67a0-1771-4442-8fd2-c3617bff23f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:11:31.283002 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1337/2644] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:11:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:11:31.664477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03b77b-be43-45bb-8f2e-0968dfc5d370 service placement] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:11:31.677971 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03b77b-be43-45bb-8f2e-0968dfc5d370 service placement] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:11:31.683306 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e03f3a-f518-4e50-89b4-33dce0fe6fed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:11:31.684012 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e03f3a-f518-4e50-89b4-33dce0fe6fed service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:11:31.684599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e03f3a-f518-4e50-89b4-33dce0fe6fed service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:11:31.684891 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e03f3a-f518-4e50-89b4-33dce0fe6fed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:11:31.687830 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b7e03f3a-f518-4e50-89b4-33dce0fe6fed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:11:31.710819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e03f3a-f518-4e50-89b4-33dce0fe6fed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:11:31.712449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e03f3a-f518-4e50-89b4-33dce0fe6fed service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:11:31.724128 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1308/2645] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:11:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:11:32.109510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae67a0-1771-4442-8fd2-c3617bff23f8 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:11:32.123280 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e03f3a-f518-4e50-89b4-33dce0fe6fed service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:11:32.135851 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae67a0-1771-4442-8fd2-c3617bff23f8 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:11:32.138382 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e03f3a-f518-4e50-89b4-33dce0fe6fed service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:11:32.140817 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e726cc-f82f-4fab-818d-35bebbb1c8c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:11:32.142107 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e726cc-f82f-4fab-818d-35bebbb1c8c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:11:32.142446 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e726cc-f82f-4fab-818d-35bebbb1c8c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:11:32.142963 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e726cc-f82f-4fab-818d-35bebbb1c8c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:11:32.144797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7cdb6-b6d4-4bf4-a94c-296edf09e9aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c1c13515a8b41a781478dc65e6a88b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:11:32.145322 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-26e726cc-f82f-4fab-818d-35bebbb1c8c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:11:32.145455 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7cdb6-b6d4-4bf4-a94c-296edf09e9aa service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:11:32.145739 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7cdb6-b6d4-4bf4-a94c-296edf09e9aa service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:11:32.146008 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7cdb6-b6d4-4bf4-a94c-296edf09e9aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:11:32.148067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0ed7cdb6-b6d4-4bf4-a94c-296edf09e9aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:11:32.162424 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed7cdb6-b6d4-4bf4-a94c-296edf09e9aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:11:32.163329 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e726cc-f82f-4fab-818d-35bebbb1c8c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:11:32.164068 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed7cdb6-b6d4-4bf4-a94c-296edf09e9aa service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:11:32.165217 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e726cc-f82f-4fab-818d-35bebbb1c8c9 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:11:32.175539 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1309/2646] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:11:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 15:11:32.180091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1338/2647] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:11:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:11:33.240844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7cdb6-b6d4-4bf4-a94c-296edf09e9aa service placement] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:11:33.242339 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0ed7cdb6-b6d4-4bf4-a94c-296edf09e9aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:11:33.261379 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7cdb6-b6d4-4bf4-a94c-296edf09e9aa service placement] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:11:33.283224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13e7a9-3712-4f10-807a-254e810bbe53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:11:33.284740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13e7a9-3712-4f10-807a-254e810bbe53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:11:33.285198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13e7a9-3712-4f10-807a-254e810bbe53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:11:33.285661 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13e7a9-3712-4f10-807a-254e810bbe53 admin admin] PATH_INFO: `/v3/users/95d4208cdd30408bae5bcdc4d9941cde` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:11:33.289799 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13e7a9-3712-4f10-807a-254e810bbe53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95d4208cdd30408bae5bcdc4d9941cde)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:11:33.292411 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13e7a9-3712-4f10-807a-254e810bbe53 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:11:33.380867 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cf13e7a9-3712-4f10-807a-254e810bbe53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95d4208cdd30408bae5bcdc4d9941cde'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:11:33.394384 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cf13e7a9-3712-4f10-807a-254e810bbe53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95d4208cdd30408bae5bcdc4d9941cde'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:11:33.402324 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cf13e7a9-3712-4f10-807a-254e810bbe53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95d4208cdd30408bae5bcdc4d9941cde'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:11:33.415211 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cf13e7a9-3712-4f10-807a-254e810bbe53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95d4208cdd30408bae5bcdc4d9941cde'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:11:33.415904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cf13e7a9-3712-4f10-807a-254e810bbe53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95d4208cdd30408bae5bcdc4d9941cde'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:11:33.422417 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1310/2648] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:11:33 2022] DELETE /identity/v3/users/95d4208cdd30408bae5bcdc4d9941cde => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:11:33.603811 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e726cc-f82f-4fab-818d-35bebbb1c8c9 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:11:33.616257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e726cc-f82f-4fab-818d-35bebbb1c8c9 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:11:33.679755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049046a6-d9bb-4c8e-bed3-9feb82a6a0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:11:33.680628 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049046a6-d9bb-4c8e-bed3-9feb82a6a0d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:11:33.680911 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049046a6-d9bb-4c8e-bed3-9feb82a6a0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:11:33.681185 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049046a6-d9bb-4c8e-bed3-9feb82a6a0d8 admin admin] PATH_INFO: `/v3/projects/21c11689a303465495b9209f70dfb664` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:11:33.682375 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049046a6-d9bb-4c8e-bed3-9feb82a6a0d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21c11689a303465495b9209f70dfb664)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:11:33.684380 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049046a6-d9bb-4c8e-bed3-9feb82a6a0d8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:11:33.733576 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-049046a6-d9bb-4c8e-bed3-9feb82a6a0d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21c11689a303465495b9209f70dfb664', 'request_id': 'req-049046a6-d9bb-4c8e-bed3-9feb82a6a0d8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:11:33.739966 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-049046a6-d9bb-4c8e-bed3-9feb82a6a0d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21c11689a303465495b9209f70dfb664', 'request_id': 'req-049046a6-d9bb-4c8e-bed3-9feb82a6a0d8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:11:33.746586 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-049046a6-d9bb-4c8e-bed3-9feb82a6a0d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21c11689a303465495b9209f70dfb664', 'request_id': 'req-049046a6-d9bb-4c8e-bed3-9feb82a6a0d8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:11:33.748352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:11:33.763582 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-049046a6-d9bb-4c8e-bed3-9feb82a6a0d8 admin admin] The token cache is being invalidate because project 21c11689a303465495b9209f70dfb664 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:11:33.763965 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-049046a6-d9bb-4c8e-bed3-9feb82a6a0d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:11:33.764710 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-049046a6-d9bb-4c8e-bed3-9feb82a6a0d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:11:33.767887 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1339/2649] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:11:33 2022] DELETE /identity/v3/projects/21c11689a303465495b9209f70dfb664 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:11:33.771863 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13e7a9-3712-4f10-807a-254e810bbe53 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:11:33.773375 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cf13e7a9-3712-4f10-807a-254e810bbe53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:11:33.787080 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13e7a9-3712-4f10-807a-254e810bbe53 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:11:33.859541 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c83756-df68-4d8b-9e4f-5f026c2684a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:11:33.860443 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c83756-df68-4d8b-9e4f-5f026c2684a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:11:33.860727 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c83756-df68-4d8b-9e4f-5f026c2684a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:11:33.861009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c83756-df68-4d8b-9e4f-5f026c2684a8 admin admin] PATH_INFO: `/v3/users/11f1f384f2a241de835900a7fef57731` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:11:33.877505 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:11:33.881457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c83756-df68-4d8b-9e4f-5f026c2684a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11f1f384f2a241de835900a7fef57731)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:11:33.883511 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c83756-df68-4d8b-9e4f-5f026c2684a8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:11:33.952123 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-36c83756-df68-4d8b-9e4f-5f026c2684a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11f1f384f2a241de835900a7fef57731'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:11:33.963203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-36c83756-df68-4d8b-9e4f-5f026c2684a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11f1f384f2a241de835900a7fef57731'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:11:33.973789 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-36c83756-df68-4d8b-9e4f-5f026c2684a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11f1f384f2a241de835900a7fef57731'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:11:33.989714 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-36c83756-df68-4d8b-9e4f-5f026c2684a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11f1f384f2a241de835900a7fef57731'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:11:33.990755 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-36c83756-df68-4d8b-9e4f-5f026c2684a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11f1f384f2a241de835900a7fef57731'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:11:34.001254 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1311/2650] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:11:33 2022] DELETE /identity/v3/users/11f1f384f2a241de835900a7fef57731 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:11:34.221032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049046a6-d9bb-4c8e-bed3-9feb82a6a0d8 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:11:34.222528 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-049046a6-d9bb-4c8e-bed3-9feb82a6a0d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:11:34.239885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049046a6-d9bb-4c8e-bed3-9feb82a6a0d8 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:11:34.307614 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6343da4a-5449-43ed-8e78-6880ecc13cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:11:34.309066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343da4a-5449-43ed-8e78-6880ecc13cba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:11:34.309358 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343da4a-5449-43ed-8e78-6880ecc13cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:11:34.309634 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343da4a-5449-43ed-8e78-6880ecc13cba admin admin] PATH_INFO: `/v3/projects/797ac2d6a8344b7ba939e3b078f44b0b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:11:34.322790 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6343da4a-5449-43ed-8e78-6880ecc13cba admin admin] RBAC: Authorizing `identity:delete_project(project_id=797ac2d6a8344b7ba939e3b078f44b0b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:11:34.325667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6343da4a-5449-43ed-8e78-6880ecc13cba admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:11:34.390589 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6343da4a-5449-43ed-8e78-6880ecc13cba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '797ac2d6a8344b7ba939e3b078f44b0b', 'request_id': 'req-6343da4a-5449-43ed-8e78-6880ecc13cba'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:11:34.398675 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6343da4a-5449-43ed-8e78-6880ecc13cba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '797ac2d6a8344b7ba939e3b078f44b0b', 'request_id': 'req-6343da4a-5449-43ed-8e78-6880ecc13cba'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:11:34.406030 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6343da4a-5449-43ed-8e78-6880ecc13cba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '797ac2d6a8344b7ba939e3b078f44b0b', 'request_id': 'req-6343da4a-5449-43ed-8e78-6880ecc13cba'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:11:34.427000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6343da4a-5449-43ed-8e78-6880ecc13cba admin admin] The token cache is being invalidate because project 797ac2d6a8344b7ba939e3b078f44b0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:11:34.427355 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6343da4a-5449-43ed-8e78-6880ecc13cba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:11:34.428524 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6343da4a-5449-43ed-8e78-6880ecc13cba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:11:34.429811 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1340/2651] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:11:34 2022] DELETE /identity/v3/projects/797ac2d6a8344b7ba939e3b078f44b0b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:11:41.803735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c83756-df68-4d8b-9e4f-5f026c2684a8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:11:41.805209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-36c83756-df68-4d8b-9e4f-5f026c2684a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:11:41.884725 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c83756-df68-4d8b-9e4f-5f026c2684a8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:11:41.946016 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b6cf8-76ea-469b-a141-f695ef61feaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:11:41.946718 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b6cf8-76ea-469b-a141-f695ef61feaa service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:11:41.947000 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b6cf8-76ea-469b-a141-f695ef61feaa service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:11:41.947274 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b6cf8-76ea-469b-a141-f695ef61feaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:11:41.949084 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7a7b6cf8-76ea-469b-a141-f695ef61feaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:11:41.962971 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b6cf8-76ea-469b-a141-f695ef61feaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:11:41.965073 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b6cf8-76ea-469b-a141-f695ef61feaa service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:11:41.970985 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:11:41.977791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1312/2652] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:11:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:11:44.423667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=57 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:11:46.543980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6343da4a-5449-43ed-8e78-6880ecc13cba admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:11:46.550619 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6343da4a-5449-43ed-8e78-6880ecc13cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:11:46.574953 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6343da4a-5449-43ed-8e78-6880ecc13cba admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:11:46.584292 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3646b-d881-4b4d-9a75-a811162ad588 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:11:46.585278 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3646b-d881-4b4d-9a75-a811162ad588 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:11:46.585707 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3646b-d881-4b4d-9a75-a811162ad588 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:11:46.586124 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3646b-d881-4b4d-9a75-a811162ad588 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:11:46.588870 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f9c3646b-d881-4b4d-9a75-a811162ad588 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:11:46.612437 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3646b-d881-4b4d-9a75-a811162ad588 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:11:46.615710 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3646b-d881-4b4d-9a75-a811162ad588 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:11:46.635154 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1341/2653] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:11:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:11:56.626278 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=6 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:12:09.336735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b6cf8-76ea-469b-a141-f695ef61feaa service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:12:09.338168 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7a7b6cf8-76ea-469b-a141-f695ef61feaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:12:09.461830 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b6cf8-76ea-469b-a141-f695ef61feaa service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:12:09.467373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262635b-a07e-4c4a-a6e9-5b453e53ae5a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c1c13515a8b41a781478dc65e6a88b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:12:09.468020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262635b-a07e-4c4a-a6e9-5b453e53ae5a service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:12:09.468307 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262635b-a07e-4c4a-a6e9-5b453e53ae5a service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:12:09.468588 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262635b-a07e-4c4a-a6e9-5b453e53ae5a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:12:09.470409 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f262635b-a07e-4c4a-a6e9-5b453e53ae5a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:12:09.482844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262635b-a07e-4c4a-a6e9-5b453e53ae5a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:12:09.484429 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262635b-a07e-4c4a-a6e9-5b453e53ae5a service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:12:09.496923 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1313/2654] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:12:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:12:19.497179 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=54 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:13:01.664668 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3646b-d881-4b4d-9a75-a811162ad588 service nova] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:13:01.714637 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3646b-d881-4b4d-9a75-a811162ad588 service nova] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:13:01.720675 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d23e69-a5da-4dd4-8451-078d7293391c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:13:01.721352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d23e69-a5da-4dd4-8451-078d7293391c service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:13:01.721639 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d23e69-a5da-4dd4-8451-078d7293391c service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:13:01.721918 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d23e69-a5da-4dd4-8451-078d7293391c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:13:01.723840 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-79d23e69-a5da-4dd4-8451-078d7293391c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:13:01.830248 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d23e69-a5da-4dd4-8451-078d7293391c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:13:01.831837 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d23e69-a5da-4dd4-8451-078d7293391c service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:13:01.854270 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1342/2655] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:13:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:13:11.851314 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=59 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:14:14.955819 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262635b-a07e-4c4a-a6e9-5b453e53ae5a service placement] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:14:14.957349 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f262635b-a07e-4c4a-a6e9-5b453e53ae5a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:14:15.052565 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262635b-a07e-4c4a-a6e9-5b453e53ae5a service placement] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:14:15.056860 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a9a3d-2044-4c5c-b607-4815a7d5ae1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:14:15.057649 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a9a3d-2044-4c5c-b607-4815a7d5ae1d service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:14:15.057973 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a9a3d-2044-4c5c-b607-4815a7d5ae1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:14:15.058301 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a9a3d-2044-4c5c-b607-4815a7d5ae1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:14:15.060536 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d95a9a3d-2044-4c5c-b607-4815a7d5ae1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:14:15.140137 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a9a3d-2044-4c5c-b607-4815a7d5ae1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:14:15.141669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a9a3d-2044-4c5c-b607-4815a7d5ae1d service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:14:15.152524 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1314/2656] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:14:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:14:25.147881 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=57 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:15:08.054580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d23e69-a5da-4dd4-8451-078d7293391c service nova] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:15:08.056224 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-79d23e69-a5da-4dd4-8451-078d7293391c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:15:08.138845 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d23e69-a5da-4dd4-8451-078d7293391c service nova] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:15:08.142981 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c2e61-8db4-49c0-80a5-a890345db4db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:15:08.143657 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c2e61-8db4-49c0-80a5-a890345db4db service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:15:08.143947 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c2e61-8db4-49c0-80a5-a890345db4db service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:15:08.144293 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c2e61-8db4-49c0-80a5-a890345db4db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:15:08.146285 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-035c2e61-8db4-49c0-80a5-a890345db4db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:15:08.157403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c2e61-8db4-49c0-80a5-a890345db4db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:15:08.159239 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c2e61-8db4-49c0-80a5-a890345db4db service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:15:08.169368 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1343/2657] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:15:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:15:18.164927 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=30 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:15:38.213172 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a9a3d-2044-4c5c-b607-4815a7d5ae1d service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:15:38.236828 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a9a3d-2044-4c5c-b607-4815a7d5ae1d service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:15:38.241605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb14bc-42e5-449c-b2dc-1221efe75b4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:15:38.242278 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb14bc-42e5-449c-b2dc-1221efe75b4d service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:15:38.242568 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb14bc-42e5-449c-b2dc-1221efe75b4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:15:38.242889 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb14bc-42e5-449c-b2dc-1221efe75b4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:15:38.244802 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-62fb14bc-42e5-449c-b2dc-1221efe75b4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:15:38.350150 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb14bc-42e5-449c-b2dc-1221efe75b4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:15:38.353001 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb14bc-42e5-449c-b2dc-1221efe75b4d service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:15:38.379210 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1315/2658] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:15:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:15:48.376094 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=45 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:17:13.011359 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c2e61-8db4-49c0-80a5-a890345db4db service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:17:13.013124 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-035c2e61-8db4-49c0-80a5-a890345db4db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:17:13.030518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c2e61-8db4-49c0-80a5-a890345db4db service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:17:13.036300 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749807d2-6c91-4cac-98f4-38bda3a21c44 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c1c13515a8b41a781478dc65e6a88b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:17:13.037074 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749807d2-6c91-4cac-98f4-38bda3a21c44 service placement] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:17:13.037397 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749807d2-6c91-4cac-98f4-38bda3a21c44 service placement] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:17:13.037719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749807d2-6c91-4cac-98f4-38bda3a21c44 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:17:13.048643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749807d2-6c91-4cac-98f4-38bda3a21c44 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:17:13.050408 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749807d2-6c91-4cac-98f4-38bda3a21c44 service placement] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:17:13.063404 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1344/2659] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:17:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:17:23.057329 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=6 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:18:01.761498 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb14bc-42e5-449c-b2dc-1221efe75b4d service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:18:01.763487 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-62fb14bc-42e5-449c-b2dc-1221efe75b4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:18:01.789470 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb14bc-42e5-449c-b2dc-1221efe75b4d service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:18:01.793209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42565bab-d8ee-42d1-907a-a4d46fb227a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:18:01.793862 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42565bab-d8ee-42d1-907a-a4d46fb227a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:18:01.794147 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42565bab-d8ee-42d1-907a-a4d46fb227a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:18:01.794423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42565bab-d8ee-42d1-907a-a4d46fb227a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:18:01.804358 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42565bab-d8ee-42d1-907a-a4d46fb227a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:18:01.805800 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42565bab-d8ee-42d1-907a-a4d46fb227a7 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:18:01.816313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1316/2660] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:18:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:18:11.811884 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=11 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:18:12.520818 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749807d2-6c91-4cac-98f4-38bda3a21c44 service placement] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:18:12.539063 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749807d2-6c91-4cac-98f4-38bda3a21c44 service placement] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:18:12.545427 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952be96-050e-47bd-b492-fd0a3e8a6ca3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:18:12.546549 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952be96-050e-47bd-b492-fd0a3e8a6ca3 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:18:12.547074 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952be96-050e-47bd-b492-fd0a3e8a6ca3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:18:12.547577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952be96-050e-47bd-b492-fd0a3e8a6ca3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:18:12.551071 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3952be96-050e-47bd-b492-fd0a3e8a6ca3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:18:12.577683 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952be96-050e-47bd-b492-fd0a3e8a6ca3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:18:12.580128 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952be96-050e-47bd-b492-fd0a3e8a6ca3 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:18:12.620527 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1345/2661] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:18:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:18:21.272784 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42565bab-d8ee-42d1-907a-a4d46fb227a7 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:18:21.286503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42565bab-d8ee-42d1-907a-a4d46fb227a7 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:18:21.290334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c371ad-e818-4de0-84f3-69b0f74f30b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:18:21.291082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c371ad-e818-4de0-84f3-69b0f74f30b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:18:21.291378 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c371ad-e818-4de0-84f3-69b0f74f30b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:18:21.291659 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c371ad-e818-4de0-84f3-69b0f74f30b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:18:21.293578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-73c371ad-e818-4de0-84f3-69b0f74f30b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:18:21.308429 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c371ad-e818-4de0-84f3-69b0f74f30b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:18:21.309961 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c371ad-e818-4de0-84f3-69b0f74f30b9 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:18:21.322370 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1317/2662] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:18:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:18:22.615344 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=13 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:18:25.995816 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952be96-050e-47bd-b492-fd0a3e8a6ca3 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:18:26.030939 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952be96-050e-47bd-b492-fd0a3e8a6ca3 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:18:26.043789 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd504a-3e89-4fe2-b6df-21eb5943c412 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:18:26.046263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd504a-3e89-4fe2-b6df-21eb5943c412 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:18:26.047376 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd504a-3e89-4fe2-b6df-21eb5943c412 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:18:26.048404 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd504a-3e89-4fe2-b6df-21eb5943c412 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:18:26.052713 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-acdd504a-3e89-4fe2-b6df-21eb5943c412 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:18:26.080676 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd504a-3e89-4fe2-b6df-21eb5943c412 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:18:26.084584 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd504a-3e89-4fe2-b6df-21eb5943c412 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:18:26.150626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1346/2663] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:18:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:18:29.958428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ce88e-3cf8-4a28-8743-324ea5d81c7e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:18:29.959139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ce88e-3cf8-4a28-8743-324ea5d81c7e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:18:29.959139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ce88e-3cf8-4a28-8743-324ea5d81c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:18:29.994256 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8d7ce88e-3cf8-4a28-8743-324ea5d81c7e None None] MFA Rules not processed for user `7a8df9e916ba4542af81e2a29229c1db`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:18:29.996291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8d7ce88e-3cf8-4a28-8743-324ea5d81c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:18:30.010119 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1318/2664] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:18:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:18:30.364996 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd504a-3e89-4fe2-b6df-21eb5943c412 service nova] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:18:30.366570 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-acdd504a-3e89-4fe2-b6df-21eb5943c412 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:18:30.379225 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd504a-3e89-4fe2-b6df-21eb5943c412 service nova] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:18:30.383362 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17f9c9-c316-49e8-a35f-fa2d7ed3dd0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:18:30.384181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17f9c9-c316-49e8-a35f-fa2d7ed3dd0f service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:18:30.384657 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17f9c9-c316-49e8-a35f-fa2d7ed3dd0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:18:30.385028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17f9c9-c316-49e8-a35f-fa2d7ed3dd0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:18:30.387140 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3b17f9c9-c316-49e8-a35f-fa2d7ed3dd0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:18:30.399237 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b17f9c9-c316-49e8-a35f-fa2d7ed3dd0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:18:30.400856 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b17f9c9-c316-49e8-a35f-fa2d7ed3dd0f service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:18:30.411238 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1347/2665] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:18:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:18:31.812419 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ce88e-3cf8-4a28-8743-324ea5d81c7e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:18:31.827941 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ce88e-3cf8-4a28-8743-324ea5d81c7e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:18:31.833612 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9ec49-d675-4c84-9bfa-4b966fdfe2ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c1c13515a8b41a781478dc65e6a88b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:18:31.834303 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9ec49-d675-4c84-9bfa-4b966fdfe2ec service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:18:31.834590 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9ec49-d675-4c84-9bfa-4b966fdfe2ec service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:18:31.834911 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9ec49-d675-4c84-9bfa-4b966fdfe2ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:18:31.836927 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cca9ec49-d675-4c84-9bfa-4b966fdfe2ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:18:31.850771 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9ec49-d675-4c84-9bfa-4b966fdfe2ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:18:31.852426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9ec49-d675-4c84-9bfa-4b966fdfe2ec service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:18:31.865017 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1319/2666] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:18:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:18:40.406971 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=12 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:18:41.859669 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=18 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:18:43.966577 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17f9c9-c316-49e8-a35f-fa2d7ed3dd0f service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:18:43.968877 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3b17f9c9-c316-49e8-a35f-fa2d7ed3dd0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:18:43.987034 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17f9c9-c316-49e8-a35f-fa2d7ed3dd0f service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:18:43.995718 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b883f95-ac17-4207-aed4-ed72861166b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:18:43.996452 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b883f95-ac17-4207-aed4-ed72861166b2 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:18:43.996751 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b883f95-ac17-4207-aed4-ed72861166b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:18:43.997033 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b883f95-ac17-4207-aed4-ed72861166b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:18:44.007881 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b883f95-ac17-4207-aed4-ed72861166b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:18:44.009608 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b883f95-ac17-4207-aed4-ed72861166b2 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:18:44.022597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1348/2667] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:18:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 15:18:49.026291 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9ec49-d675-4c84-9bfa-4b966fdfe2ec service placement] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:18:49.041399 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9ec49-d675-4c84-9bfa-4b966fdfe2ec service placement] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:18:49.048733 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1dcc59-d2f3-4429-8e00-00968884da7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:18:49.049675 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1dcc59-d2f3-4429-8e00-00968884da7a service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:18:49.050132 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1dcc59-d2f3-4429-8e00-00968884da7a service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:18:49.050539 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1dcc59-d2f3-4429-8e00-00968884da7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:18:49.065016 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1dcc59-d2f3-4429-8e00-00968884da7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:18:49.067358 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1dcc59-d2f3-4429-8e00-00968884da7a service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:18:49.082138 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1320/2668] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:18:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:18:54.018432 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=6 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:18:59.075218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=6 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:19:14.840655 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b883f95-ac17-4207-aed4-ed72861166b2 service nova] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:14.852225 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b883f95-ac17-4207-aed4-ed72861166b2 service nova] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:14.856677 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c4030-ea83-42ea-abf0-ce79c1dad4a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:14.857325 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c4030-ea83-42ea-abf0-ce79c1dad4a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:14.857607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c4030-ea83-42ea-abf0-ce79c1dad4a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:14.857881 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c4030-ea83-42ea-abf0-ce79c1dad4a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:14.868516 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c4030-ea83-42ea-abf0-ce79c1dad4a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:14.870031 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c4030-ea83-42ea-abf0-ce79c1dad4a1 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:14.903869 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1349/2669] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:19:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:19:24.900066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=13 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:19:27.739159 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31412ae-8f07-4c33-9e50-6cc33d884c61 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:27.739745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31412ae-8f07-4c33-9e50-6cc33d884c61 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:27.739745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31412ae-8f07-4c33-9e50-6cc33d884c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:27.806027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e31412ae-8f07-4c33-9e50-6cc33d884c61 None None] MFA Rules not processed for user `0b7d100f2407432381d119c25814fff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:19:27.807298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e31412ae-8f07-4c33-9e50-6cc33d884c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:27.881745 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1321/2670] 10.222.0.67 () {58 vars in 967 bytes} [Tue May 31 15:19:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:19:27.934153 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c4030-ea83-42ea-abf0-ce79c1dad4a1 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:27.936859 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3e4c4030-ea83-42ea-abf0-ce79c1dad4a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:28.086240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c4030-ea83-42ea-abf0-ce79c1dad4a1 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:28.093657 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3409f-c5b9-41da-9945-8232bdd1fff7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfa2282f957941b98db1d7497da3351b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:28.094350 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3409f-c5b9-41da-9945-8232bdd1fff7 service swift] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:28.094667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3409f-c5b9-41da-9945-8232bdd1fff7 service swift] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:28.094964 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3409f-c5b9-41da-9945-8232bdd1fff7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:28.116827 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3409f-c5b9-41da-9945-8232bdd1fff7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:28.119624 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3409f-c5b9-41da-9945-8232bdd1fff7 service swift] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:28.132685 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1350/2671] 10.222.0.67 () {60 vars in 1384 bytes} [Tue May 31 15:19:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 31 15:19:28.735264 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31412ae-8f07-4c33-9e50-6cc33d884c61 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:28.746633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31412ae-8f07-4c33-9e50-6cc33d884c61 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:28.752176 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b1594-9ad2-404f-809b-b2568ba4d875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:28.752868 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b1594-9ad2-404f-809b-b2568ba4d875 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:28.753159 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b1594-9ad2-404f-809b-b2568ba4d875 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:28.753438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b1594-9ad2-404f-809b-b2568ba4d875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:28.755656 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-134b1594-9ad2-404f-809b-b2568ba4d875 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:28.873743 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b1594-9ad2-404f-809b-b2568ba4d875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:28.875583 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b1594-9ad2-404f-809b-b2568ba4d875 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:28.898797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1322/2672] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:19:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:19:33.386993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158d98b-ce58-43bb-bead-8021d70c1878 None None] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:33.388097 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158d98b-ce58-43bb-bead-8021d70c1878 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:33.388764 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158d98b-ce58-43bb-bead-8021d70c1878 None None] PATH_INFO: `/` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:33.390599 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1351/2673] 10.222.0.67 () {54 vars in 921 bytes} [Tue May 31 15:19:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 31 15:19:33.435067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b086b-a6f7-4474-ac5c-a3e84a9e3ad6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:33.435849 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b086b-a6f7-4474-ac5c-a3e84a9e3ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:33.435989 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b086b-a6f7-4474-ac5c-a3e84a9e3ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:33.490985 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:19:33.495285 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c38b086b-a6f7-4474-ac5c-a3e84a9e3ad6 None None] MFA Rules not processed for user `6c1c13515a8b41a781478dc65e6a88b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:19:33.497588 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c38b086b-a6f7-4474-ac5c-a3e84a9e3ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:33.508648 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1323/2674] 10.222.0.67 () {58 vars in 1018 bytes} [Tue May 31 15:19:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:19:33.532788 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158d98b-ce58-43bb-bead-8021d70c1878 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:33.548401 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158d98b-ce58-43bb-bead-8021d70c1878 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:33.553957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fec7bb-a2ba-48a5-a8e6-0e0ccbf5f283 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c1c13515a8b41a781478dc65e6a88b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:33.554622 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fec7bb-a2ba-48a5-a8e6-0e0ccbf5f283 service placement] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:33.554969 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fec7bb-a2ba-48a5-a8e6-0e0ccbf5f283 service placement] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:33.555245 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fec7bb-a2ba-48a5-a8e6-0e0ccbf5f283 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:33.566518 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fec7bb-a2ba-48a5-a8e6-0e0ccbf5f283 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:33.569504 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fec7bb-a2ba-48a5-a8e6-0e0ccbf5f283 service placement] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:33.581103 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1352/2675] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:19:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:19:35.795113 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b086b-a6f7-4474-ac5c-a3e84a9e3ad6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:35.812339 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b086b-a6f7-4474-ac5c-a3e84a9e3ad6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:35.818768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027f1bd-02b0-427e-838c-c1cd0a74968e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:35.819970 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027f1bd-02b0-427e-838c-c1cd0a74968e service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:35.820469 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027f1bd-02b0-427e-838c-c1cd0a74968e service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:35.820992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027f1bd-02b0-427e-838c-c1cd0a74968e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:35.824241 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3027f1bd-02b0-427e-838c-c1cd0a74968e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:35.846672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027f1bd-02b0-427e-838c-c1cd0a74968e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:35.849667 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027f1bd-02b0-427e-838c-c1cd0a74968e service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:35.870385 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1324/2676] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:19:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:19:37.287208 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fec7bb-a2ba-48a5-a8e6-0e0ccbf5f283 service placement] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:37.308354 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fec7bb-a2ba-48a5-a8e6-0e0ccbf5f283 service placement] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:37.315719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5c561-990f-4cfc-9447-0e10885dbc04 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c1c13515a8b41a781478dc65e6a88b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:37.317007 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5c561-990f-4cfc-9447-0e10885dbc04 service placement] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:37.317350 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5c561-990f-4cfc-9447-0e10885dbc04 service placement] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:37.317765 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5c561-990f-4cfc-9447-0e10885dbc04 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:37.320865 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ffb5c561-990f-4cfc-9447-0e10885dbc04 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:37.347460 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb5c561-990f-4cfc-9447-0e10885dbc04 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:37.349644 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb5c561-990f-4cfc-9447-0e10885dbc04 service placement] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:37.369148 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027f1bd-02b0-427e-838c-c1cd0a74968e service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:37.371318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3027f1bd-02b0-427e-838c-c1cd0a74968e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:37.374334 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1353/2677] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:19:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:19:37.398485 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027f1bd-02b0-427e-838c-c1cd0a74968e service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:37.406122 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd25ae-16b1-4c0b-aeeb-b414f94583ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:37.407183 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd25ae-16b1-4c0b-aeeb-b414f94583ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:37.407660 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd25ae-16b1-4c0b-aeeb-b414f94583ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:37.408133 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd25ae-16b1-4c0b-aeeb-b414f94583ea admin admin] PATH_INFO: `/v3/users/180eebbd503e441b9332720e70f7f339` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:37.409608 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd25ae-16b1-4c0b-aeeb-b414f94583ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=180eebbd503e441b9332720e70f7f339)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:37.411721 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd25ae-16b1-4c0b-aeeb-b414f94583ea admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:37.488568 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-47dd25ae-16b1-4c0b-aeeb-b414f94583ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '180eebbd503e441b9332720e70f7f339'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:37.501953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-47dd25ae-16b1-4c0b-aeeb-b414f94583ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '180eebbd503e441b9332720e70f7f339'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:37.511185 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-47dd25ae-16b1-4c0b-aeeb-b414f94583ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '180eebbd503e441b9332720e70f7f339'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:37.524354 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-47dd25ae-16b1-4c0b-aeeb-b414f94583ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '180eebbd503e441b9332720e70f7f339'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:37.525110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-47dd25ae-16b1-4c0b-aeeb-b414f94583ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '180eebbd503e441b9332720e70f7f339'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:37.532986 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1325/2678] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:19:37 2022] DELETE /identity/v3/users/180eebbd503e441b9332720e70f7f339 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 15:19:37.751750 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5c561-990f-4cfc-9447-0e10885dbc04 service placement] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:37.781337 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5c561-990f-4cfc-9447-0e10885dbc04 service placement] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:37.884355 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1222ed-31e2-4c5e-ae2d-e8686803cc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:37.886881 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1222ed-31e2-4c5e-ae2d-e8686803cc55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:37.887467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1222ed-31e2-4c5e-ae2d-e8686803cc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:37.888039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1222ed-31e2-4c5e-ae2d-e8686803cc55 admin admin] PATH_INFO: `/v3/projects/f24fdc4bc9ca442e948484e73713779c` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:37.890965 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1222ed-31e2-4c5e-ae2d-e8686803cc55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f24fdc4bc9ca442e948484e73713779c)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:37.895427 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1222ed-31e2-4c5e-ae2d-e8686803cc55 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:37.968613 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:19:37.973616 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4f1222ed-31e2-4c5e-ae2d-e8686803cc55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f24fdc4bc9ca442e948484e73713779c', 'request_id': 'req-4f1222ed-31e2-4c5e-ae2d-e8686803cc55'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:37.989238 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4f1222ed-31e2-4c5e-ae2d-e8686803cc55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f24fdc4bc9ca442e948484e73713779c', 'request_id': 'req-4f1222ed-31e2-4c5e-ae2d-e8686803cc55'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:38.006933 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4f1222ed-31e2-4c5e-ae2d-e8686803cc55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f24fdc4bc9ca442e948484e73713779c', 'request_id': 'req-4f1222ed-31e2-4c5e-ae2d-e8686803cc55'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:38.032136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4f1222ed-31e2-4c5e-ae2d-e8686803cc55 admin admin] The token cache is being invalidate because project f24fdc4bc9ca442e948484e73713779c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:19:38.032790 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4f1222ed-31e2-4c5e-ae2d-e8686803cc55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:38.034831 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-4f1222ed-31e2-4c5e-ae2d-e8686803cc55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:38.037471 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1354/2679] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:19:37 2022] DELETE /identity/v3/projects/f24fdc4bc9ca442e948484e73713779c => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 15:19:38.041925 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd25ae-16b1-4c0b-aeeb-b414f94583ea admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:38.043927 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-47dd25ae-16b1-4c0b-aeeb-b414f94583ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:38.058892 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd25ae-16b1-4c0b-aeeb-b414f94583ea admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:38.151828 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4f853-6c09-4d25-a33c-78f5537d0cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:38.153100 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4f853-6c09-4d25-a33c-78f5537d0cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:38.153440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4f853-6c09-4d25-a33c-78f5537d0cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:38.153783 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4f853-6c09-4d25-a33c-78f5537d0cb9 admin admin] PATH_INFO: `/v3/users/8a00d14831a34f5ab9e455a327b01491` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:38.171459 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4f853-6c09-4d25-a33c-78f5537d0cb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a00d14831a34f5ab9e455a327b01491)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:38.173582 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4f853-6c09-4d25-a33c-78f5537d0cb9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:38.242598 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cae4f853-6c09-4d25-a33c-78f5537d0cb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a00d14831a34f5ab9e455a327b01491'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:38.248156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:19:38.255571 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cae4f853-6c09-4d25-a33c-78f5537d0cb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a00d14831a34f5ab9e455a327b01491'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:38.269516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cae4f853-6c09-4d25-a33c-78f5537d0cb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a00d14831a34f5ab9e455a327b01491'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:38.285982 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cae4f853-6c09-4d25-a33c-78f5537d0cb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a00d14831a34f5ab9e455a327b01491'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:38.286778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cae4f853-6c09-4d25-a33c-78f5537d0cb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a00d14831a34f5ab9e455a327b01491'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:38.293752 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1326/2680] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:19:38 2022] DELETE /identity/v3/users/8a00d14831a34f5ab9e455a327b01491 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 15:19:38.503281 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1222ed-31e2-4c5e-ae2d-e8686803cc55 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:38.504856 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4f1222ed-31e2-4c5e-ae2d-e8686803cc55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:38.522046 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1222ed-31e2-4c5e-ae2d-e8686803cc55 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:38.589768 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c30406-99cc-40c3-85f8-824ac8549b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:38.591245 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c30406-99cc-40c3-85f8-824ac8549b63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:38.591773 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c30406-99cc-40c3-85f8-824ac8549b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:38.593000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c30406-99cc-40c3-85f8-824ac8549b63 admin admin] PATH_INFO: `/v3/projects/6b2fe1e09b7e4b43a978a345e7566a3a` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:38.609529 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c30406-99cc-40c3-85f8-824ac8549b63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b2fe1e09b7e4b43a978a345e7566a3a)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:38.611531 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c30406-99cc-40c3-85f8-824ac8549b63 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:38.660793 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e5c30406-99cc-40c3-85f8-824ac8549b63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b2fe1e09b7e4b43a978a345e7566a3a', 'request_id': 'req-e5c30406-99cc-40c3-85f8-824ac8549b63'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:38.667208 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e5c30406-99cc-40c3-85f8-824ac8549b63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b2fe1e09b7e4b43a978a345e7566a3a', 'request_id': 'req-e5c30406-99cc-40c3-85f8-824ac8549b63'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:38.678372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e5c30406-99cc-40c3-85f8-824ac8549b63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b2fe1e09b7e4b43a978a345e7566a3a', 'request_id': 'req-e5c30406-99cc-40c3-85f8-824ac8549b63'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:38.693744 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e5c30406-99cc-40c3-85f8-824ac8549b63 admin admin] The token cache is being invalidate because project 6b2fe1e09b7e4b43a978a345e7566a3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:19:38.694382 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e5c30406-99cc-40c3-85f8-824ac8549b63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:38.695156 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e5c30406-99cc-40c3-85f8-824ac8549b63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:38.696865 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1355/2681] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:19:38 2022] DELETE /identity/v3/projects/6b2fe1e09b7e4b43a978a345e7566a3a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:19:38.703850 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6fba23-76a9-4797-9c86-029f4ae8a76a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:38.703850 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6fba23-76a9-4797-9c86-029f4ae8a76a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:38.703850 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6fba23-76a9-4797-9c86-029f4ae8a76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:38.757863 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-df6fba23-76a9-4797-9c86-029f4ae8a76a None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:19:38.758637 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-df6fba23-76a9-4797-9c86-029f4ae8a76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:38.857100 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1327/2682] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:19:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 15:19:38.859196 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e7a2f-ed94-4b47-91c8-4fdf92a7340f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:38.859612 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e7a2f-ed94-4b47-91c8-4fdf92a7340f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:38.860002 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e7a2f-ed94-4b47-91c8-4fdf92a7340f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:38.881707 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-c32e7a2f-ed94-4b47-91c8-4fdf92a7340f None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:19:38.883241 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c32e7a2f-ed94-4b47-91c8-4fdf92a7340f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:38.893922 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1356/2683] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:19:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:19:38.898358 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6fba23-76a9-4797-9c86-029f4ae8a76a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:38.921657 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6fba23-76a9-4797-9c86-029f4ae8a76a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:38.927318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a70137-3fcc-4b80-975f-a77619576b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:38.928063 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a70137-3fcc-4b80-975f-a77619576b32 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:38.928370 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a70137-3fcc-4b80-975f-a77619576b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:38.928660 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a70137-3fcc-4b80-975f-a77619576b32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:38.929447 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a70137-3fcc-4b80-975f-a77619576b32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 15:19:38.929764 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a70137-3fcc-4b80-975f-a77619576b32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:38.934716 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a70137-3fcc-4b80-975f-a77619576b32 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:38.949651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1328/2684] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 15:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 15:19:38.954182 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e7a2f-ed94-4b47-91c8-4fdf92a7340f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:38.968591 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e7a2f-ed94-4b47-91c8-4fdf92a7340f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:38.973697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc8f94-cd20-492b-921d-bfc08b1ea55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:38.974430 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc8f94-cd20-492b-921d-bfc08b1ea55a admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:38.974758 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc8f94-cd20-492b-921d-bfc08b1ea55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:38.975732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc8f94-cd20-492b-921d-bfc08b1ea55a admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:38.976946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfc8f94-cd20-492b-921d-bfc08b1ea55a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:38.978892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfc8f94-cd20-492b-921d-bfc08b1ea55a admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:39.026692 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1357/2685] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 15:19:38 2022] POST /identity/v3/projects => generated 371 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 15:19:39.029693 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a70137-3fcc-4b80-975f-a77619576b32 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:39.040782 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a70137-3fcc-4b80-975f-a77619576b32 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:39.119134 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4b5dd-8d83-4ca4-9d1d-ed60baa8f72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:39.120513 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4b5dd-8d83-4ca4-9d1d-ed60baa8f72f admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:39.120799 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4b5dd-8d83-4ca4-9d1d-ed60baa8f72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:39.121074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4b5dd-8d83-4ca4-9d1d-ed60baa8f72f admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:39.121785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4b5dd-8d83-4ca4-9d1d-ed60baa8f72f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:39.123785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4b5dd-8d83-4ca4-9d1d-ed60baa8f72f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:39.148512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1329/2686] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 15:19:39 2022] POST /identity/v3/users => generated 286 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 15:19:39.153453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc8f94-cd20-492b-921d-bfc08b1ea55a admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:39.170826 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc8f94-cd20-492b-921d-bfc08b1ea55a admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:39.177891 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f432ca-c9b3-4fd6-b669-a836abc6461a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:39.178605 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f432ca-c9b3-4fd6-b669-a836abc6461a admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:39.178919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f432ca-c9b3-4fd6-b669-a836abc6461a admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:39.179200 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f432ca-c9b3-4fd6-b669-a836abc6461a admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:39.179978 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f432ca-c9b3-4fd6-b669-a836abc6461a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:39.181635 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f432ca-c9b3-4fd6-b669-a836abc6461a admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:39.194425 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1358/2687] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:19:39.198726 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4b5dd-8d83-4ca4-9d1d-ed60baa8f72f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:39.211511 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4b5dd-8d83-4ca4-9d1d-ed60baa8f72f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:39.217376 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f60c5f-e024-4a5d-99f6-997cb5336da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:39.218015 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f60c5f-e024-4a5d-99f6-997cb5336da2 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:39.218316 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f60c5f-e024-4a5d-99f6-997cb5336da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:39.218579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f60c5f-e024-4a5d-99f6-997cb5336da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:39.219378 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f60c5f-e024-4a5d-99f6-997cb5336da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:39.221020 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f60c5f-e024-4a5d-99f6-997cb5336da2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:39.236810 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1330/2688] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:19:39.242444 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f432ca-c9b3-4fd6-b669-a836abc6461a admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:39.270915 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f432ca-c9b3-4fd6-b669-a836abc6461a admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:39.292126 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abf9fd-e4f3-4f3f-829d-9c9e2f2a11ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:39.293175 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abf9fd-e4f3-4f3f-829d-9c9e2f2a11ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:39.293540 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abf9fd-e4f3-4f3f-829d-9c9e2f2a11ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:39.293911 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abf9fd-e4f3-4f3f-829d-9c9e2f2a11ce admin admin] PATH_INFO: `/v3/projects/e24193a7426547f5895f8b8074e46875/users/6a8c202f3b4a424cb6565fc8a6abd15b/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:39.297333 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:19:39.323585 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1abf9fd-e4f3-4f3f-829d-9c9e2f2a11ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=e24193a7426547f5895f8b8074e46875, user_id=6a8c202f3b4a424cb6565fc8a6abd15b, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:39.327209 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1abf9fd-e4f3-4f3f-829d-9c9e2f2a11ce admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:39.348256 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1359/2689] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:19:39 2022] PUT /identity/v3/projects/e24193a7426547f5895f8b8074e46875/users/6a8c202f3b4a424cb6565fc8a6abd15b/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:19:39.354330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e55bda-3e3a-4c10-bc36-7cf138caca24 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:39.354724 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e55bda-3e3a-4c10-bc36-7cf138caca24 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:39.355113 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e55bda-3e3a-4c10-bc36-7cf138caca24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:39.376748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f4e55bda-3e3a-4c10-bc36-7cf138caca24 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:19:39.378214 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f4e55bda-3e3a-4c10-bc36-7cf138caca24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:39.384789 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:19:39.466834 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1331/2690] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:19:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:19:39.486509 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abf9fd-e4f3-4f3f-829d-9c9e2f2a11ce admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:39.492500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b1abf9fd-e4f3-4f3f-829d-9c9e2f2a11ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:39.559158 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abf9fd-e4f3-4f3f-829d-9c9e2f2a11ce admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:39.563996 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb77e11-b96e-403b-b448-29a3178514c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:39.565066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb77e11-b96e-403b-b448-29a3178514c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:39.565066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb77e11-b96e-403b-b448-29a3178514c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:39.566196 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb77e11-b96e-403b-b448-29a3178514c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:39.577749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb77e11-b96e-403b-b448-29a3178514c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:39.580559 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb77e11-b96e-403b-b448-29a3178514c1 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:39.630149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1360/2691] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:19:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:19:39.746112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e55bda-3e3a-4c10-bc36-7cf138caca24 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:39.748373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f4e55bda-3e3a-4c10-bc36-7cf138caca24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:39.770219 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e55bda-3e3a-4c10-bc36-7cf138caca24 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:39.776620 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0edf3f-86d4-4629-8ed9-af4c1a8b8849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:39.784523 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0edf3f-86d4-4629-8ed9-af4c1a8b8849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:39.784523 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0edf3f-86d4-4629-8ed9-af4c1a8b8849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:39.785348 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0edf3f-86d4-4629-8ed9-af4c1a8b8849 admin admin] PATH_INFO: `/v3/users/a36fb9afb5d54d3493e8752bae88dd13` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:39.804598 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0edf3f-86d4-4629-8ed9-af4c1a8b8849 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a36fb9afb5d54d3493e8752bae88dd13)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:39.809396 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0edf3f-86d4-4629-8ed9-af4c1a8b8849 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:39.866369 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7e0edf3f-86d4-4629-8ed9-af4c1a8b8849 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a36fb9afb5d54d3493e8752bae88dd13'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:39.875108 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7e0edf3f-86d4-4629-8ed9-af4c1a8b8849 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a36fb9afb5d54d3493e8752bae88dd13'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:39.883295 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7e0edf3f-86d4-4629-8ed9-af4c1a8b8849 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a36fb9afb5d54d3493e8752bae88dd13'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:39.899263 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7e0edf3f-86d4-4629-8ed9-af4c1a8b8849 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a36fb9afb5d54d3493e8752bae88dd13'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:39.900177 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7e0edf3f-86d4-4629-8ed9-af4c1a8b8849 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a36fb9afb5d54d3493e8752bae88dd13'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:39.911330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1332/2692] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:19:39 2022] DELETE /identity/v3/users/a36fb9afb5d54d3493e8752bae88dd13 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 15:19:40.177171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb77e11-b96e-403b-b448-29a3178514c1 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:40.190861 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb77e11-b96e-403b-b448-29a3178514c1 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:40.259615 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2e298-6b60-4127-b77a-d54ac808bbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:40.261129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2e298-6b60-4127-b77a-d54ac808bbaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:40.261415 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2e298-6b60-4127-b77a-d54ac808bbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:40.261695 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2e298-6b60-4127-b77a-d54ac808bbaa admin admin] PATH_INFO: `/v3/projects/e870af07745948ffa62b2e59eb8054f4` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:40.264270 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2e298-6b60-4127-b77a-d54ac808bbaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=e870af07745948ffa62b2e59eb8054f4)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:40.266508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2e298-6b60-4127-b77a-d54ac808bbaa admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:40.316889 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ead2e298-6b60-4127-b77a-d54ac808bbaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e870af07745948ffa62b2e59eb8054f4', 'request_id': 'req-ead2e298-6b60-4127-b77a-d54ac808bbaa'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:40.323435 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ead2e298-6b60-4127-b77a-d54ac808bbaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e870af07745948ffa62b2e59eb8054f4', 'request_id': 'req-ead2e298-6b60-4127-b77a-d54ac808bbaa'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:40.330998 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ead2e298-6b60-4127-b77a-d54ac808bbaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e870af07745948ffa62b2e59eb8054f4', 'request_id': 'req-ead2e298-6b60-4127-b77a-d54ac808bbaa'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:40.347277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ead2e298-6b60-4127-b77a-d54ac808bbaa admin admin] The token cache is being invalidate because project e870af07745948ffa62b2e59eb8054f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:19:40.347574 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ead2e298-6b60-4127-b77a-d54ac808bbaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:40.348324 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ead2e298-6b60-4127-b77a-d54ac808bbaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:40.350126 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1361/2693] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:19:40 2022] DELETE /identity/v3/projects/e870af07745948ffa62b2e59eb8054f4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:19:40.356330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87b936-415b-4170-8182-eda9dc2f2661 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:40.356618 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87b936-415b-4170-8182-eda9dc2f2661 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:40.356916 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87b936-415b-4170-8182-eda9dc2f2661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:40.380161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4e87b936-415b-4170-8182-eda9dc2f2661 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:19:40.381650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4e87b936-415b-4170-8182-eda9dc2f2661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:40.460398 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1333/2694] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:19:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:19:40.470764 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0cd8b-e200-4815-9055-24539aee6dc0 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:40.471545 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0cd8b-e200-4815-9055-24539aee6dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:40.472242 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0cd8b-e200-4815-9055-24539aee6dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:40.504805 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-75d0cd8b-e200-4815-9055-24539aee6dc0 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:19:40.506374 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-75d0cd8b-e200-4815-9055-24539aee6dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:40.517381 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1362/2695] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:19:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:19:40.522035 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87b936-415b-4170-8182-eda9dc2f2661 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:40.537956 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87b936-415b-4170-8182-eda9dc2f2661 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:40.545103 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f7cba-ec84-4b44-907b-c381cbe8d0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:40.545805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f7cba-ec84-4b44-907b-c381cbe8d0ce admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:40.546098 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f7cba-ec84-4b44-907b-c381cbe8d0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:40.546373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f7cba-ec84-4b44-907b-c381cbe8d0ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:40.547221 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f7cba-ec84-4b44-907b-c381cbe8d0ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 15:19:40.547538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f7cba-ec84-4b44-907b-c381cbe8d0ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:40.549612 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f7cba-ec84-4b44-907b-c381cbe8d0ce admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:40.565896 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:19:40.576019 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1334/2696] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 15:19:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 15:19:40.579085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0cd8b-e200-4815-9055-24539aee6dc0 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:40.592763 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0cd8b-e200-4815-9055-24539aee6dc0 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:40.605733 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6455437-5f5e-4ede-9209-c4a50f3d60d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:40.607728 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6455437-5f5e-4ede-9209-c4a50f3d60d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:40.608009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6455437-5f5e-4ede-9209-c4a50f3d60d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:40.608437 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6455437-5f5e-4ede-9209-c4a50f3d60d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:40.609557 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6455437-5f5e-4ede-9209-c4a50f3d60d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:40.613284 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6455437-5f5e-4ede-9209-c4a50f3d60d2 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:40.624342 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:19:40.653809 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1363/2697] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 15:19:40 2022] POST /identity/v3/projects => generated 383 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:19:40.659930 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f7cba-ec84-4b44-907b-c381cbe8d0ce admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:40.673089 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f7cba-ec84-4b44-907b-c381cbe8d0ce admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:40.765430 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf8fab-1d0d-46e8-b056-163094ca82c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:40.766945 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf8fab-1d0d-46e8-b056-163094ca82c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:40.767372 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf8fab-1d0d-46e8-b056-163094ca82c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:40.767793 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf8fab-1d0d-46e8-b056-163094ca82c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:40.768643 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf8fab-1d0d-46e8-b056-163094ca82c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:40.771416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf8fab-1d0d-46e8-b056-163094ca82c8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:40.813906 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6455437-5f5e-4ede-9209-c4a50f3d60d2 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:40.822999 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1335/2698] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 15:19:40 2022] POST /identity/v3/users => generated 292 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:19:40.834588 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6455437-5f5e-4ede-9209-c4a50f3d60d2 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:40.849667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554956f-aa8d-4dc0-a734-f382dcf4e55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:40.850367 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554956f-aa8d-4dc0-a734-f382dcf4e55d admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:40.850673 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554956f-aa8d-4dc0-a734-f382dcf4e55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:40.850956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554956f-aa8d-4dc0-a734-f382dcf4e55d admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:40.851874 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5554956f-aa8d-4dc0-a734-f382dcf4e55d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:40.854282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5554956f-aa8d-4dc0-a734-f382dcf4e55d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:40.873048 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1364/2699] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:19:40.877875 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf8fab-1d0d-46e8-b056-163094ca82c8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:40.892083 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf8fab-1d0d-46e8-b056-163094ca82c8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:40.897759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31088a6b-d749-4448-8a0f-a23fc65ad4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:40.898517 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31088a6b-d749-4448-8a0f-a23fc65ad4f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:40.898857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31088a6b-d749-4448-8a0f-a23fc65ad4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:40.899145 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31088a6b-d749-4448-8a0f-a23fc65ad4f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:40.899898 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31088a6b-d749-4448-8a0f-a23fc65ad4f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:40.902183 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31088a6b-d749-4448-8a0f-a23fc65ad4f8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:40.917053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1336/2700] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:19:40.921473 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554956f-aa8d-4dc0-a734-f382dcf4e55d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:40.943085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554956f-aa8d-4dc0-a734-f382dcf4e55d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:40.950908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a51a9-69bc-4a3d-9e01-7868ffc07368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:40.952441 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a51a9-69bc-4a3d-9e01-7868ffc07368 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:40.953066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a51a9-69bc-4a3d-9e01-7868ffc07368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:40.953643 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a51a9-69bc-4a3d-9e01-7868ffc07368 admin admin] PATH_INFO: `/v3/projects/a3fca267681440049f30d28fe74370b7/users/614d54ac37d743eea29632bf088e5e27/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:40.978312 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a51a9-69bc-4a3d-9e01-7868ffc07368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3fca267681440049f30d28fe74370b7, user_id=614d54ac37d743eea29632bf088e5e27, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:40.985724 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a51a9-69bc-4a3d-9e01-7868ffc07368 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:41.014607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1365/2701] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:19:40 2022] PUT /identity/v3/projects/a3fca267681440049f30d28fe74370b7/users/614d54ac37d743eea29632bf088e5e27/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:19:41.020856 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15ec68-79d0-4bdc-a993-f58c112956ba None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:41.021303 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15ec68-79d0-4bdc-a993-f58c112956ba None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:41.021731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15ec68-79d0-4bdc-a993-f58c112956ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:41.057255 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-dc15ec68-79d0-4bdc-a993-f58c112956ba None None] MFA Rules not processed for user `614d54ac37d743eea29632bf088e5e27`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:19:41.058818 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dc15ec68-79d0-4bdc-a993-f58c112956ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:41.149713 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1337/2702] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:19:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:19:41.162931 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a51a9-69bc-4a3d-9e01-7868ffc07368 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:41.169047 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ef8a51a9-69bc-4a3d-9e01-7868ffc07368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:41.273245 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a51a9-69bc-4a3d-9e01-7868ffc07368 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:41.282179 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52beae5c-9a08-4661-8c29-0ba7d6323544 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:41.282890 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52beae5c-9a08-4661-8c29-0ba7d6323544 service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:41.283166 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52beae5c-9a08-4661-8c29-0ba7d6323544 service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:41.283903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52beae5c-9a08-4661-8c29-0ba7d6323544 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:41.300236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52beae5c-9a08-4661-8c29-0ba7d6323544 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:41.302160 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52beae5c-9a08-4661-8c29-0ba7d6323544 service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:41.314135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1366/2703] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:19:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:19:44.636146 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e0d96-684f-4550-aac3-a79a6038f656 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:44.636631 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e0d96-684f-4550-aac3-a79a6038f656 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:44.636734 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e0d96-684f-4550-aac3-a79a6038f656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:44.668892 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-194e0d96-684f-4550-aac3-a79a6038f656 None None] MFA Rules not processed for user `6a8c202f3b4a424cb6565fc8a6abd15b`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:19:44.670795 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-194e0d96-684f-4550-aac3-a79a6038f656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:44.750180 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:19:44.758201 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1338/2704] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:19:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:19:44.763700 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52beae5c-9a08-4661-8c29-0ba7d6323544 service nova] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:44.765223 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-52beae5c-9a08-4661-8c29-0ba7d6323544 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:44.847977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52beae5c-9a08-4661-8c29-0ba7d6323544 service nova] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:44.855394 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4900c3-c969-4401-bf5d-5f84d1c4072d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:44.856276 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4900c3-c969-4401-bf5d-5f84d1c4072d admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:44.856670 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4900c3-c969-4401-bf5d-5f84d1c4072d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:44.857049 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4900c3-c969-4401-bf5d-5f84d1c4072d admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:44.857894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4900c3-c969-4401-bf5d-5f84d1c4072d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:44.860036 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4900c3-c969-4401-bf5d-5f84d1c4072d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:44.902823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1367/2705] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 15:19:44 2022] POST /identity/v3/projects => generated 371 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 15:19:44.903684 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e0d96-684f-4550-aac3-a79a6038f656 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:44.916987 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e0d96-684f-4550-aac3-a79a6038f656 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:45.015886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b473072-900e-4d0e-b5c9-19afdf345b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:45.016661 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b473072-900e-4d0e-b5c9-19afdf345b6a admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:45.016954 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b473072-900e-4d0e-b5c9-19afdf345b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:45.017239 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b473072-900e-4d0e-b5c9-19afdf345b6a admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:45.017962 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b473072-900e-4d0e-b5c9-19afdf345b6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:45.026783 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b473072-900e-4d0e-b5c9-19afdf345b6a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:45.056780 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1339/2706] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 15:19:44 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 15:19:45.064512 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4900c3-c969-4401-bf5d-5f84d1c4072d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:45.079466 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4900c3-c969-4401-bf5d-5f84d1c4072d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:45.084632 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3fa4f6-dbb4-4323-b1ba-dfe7a1dee4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:45.085356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fa4f6-dbb4-4323-b1ba-dfe7a1dee4c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:45.085699 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fa4f6-dbb4-4323-b1ba-dfe7a1dee4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:45.085925 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fa4f6-dbb4-4323-b1ba-dfe7a1dee4c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:45.086706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3fa4f6-dbb4-4323-b1ba-dfe7a1dee4c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:45.088942 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3fa4f6-dbb4-4323-b1ba-dfe7a1dee4c4 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:45.105370 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1368/2707] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:19:45.116025 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b473072-900e-4d0e-b5c9-19afdf345b6a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:45.134140 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b473072-900e-4d0e-b5c9-19afdf345b6a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:45.141093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bcf50-3bb2-42d1-9460-41e3a62f0774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:45.142037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bcf50-3bb2-42d1-9460-41e3a62f0774 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:45.142321 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bcf50-3bb2-42d1-9460-41e3a62f0774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:45.142602 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bcf50-3bb2-42d1-9460-41e3a62f0774 admin admin] PATH_INFO: `/v3/projects/12d69bfb660f48efb90d7fff9bea45d1/users/62c33a95fb124fb78697cfed4f1fe976/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:45.161441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953bcf50-3bb2-42d1-9460-41e3a62f0774 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12d69bfb660f48efb90d7fff9bea45d1, user_id=62c33a95fb124fb78697cfed4f1fe976, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:45.164302 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953bcf50-3bb2-42d1-9460-41e3a62f0774 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:45.181461 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1340/2708] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:19:45 2022] PUT /identity/v3/projects/12d69bfb660f48efb90d7fff9bea45d1/users/62c33a95fb124fb78697cfed4f1fe976/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:19:45.186692 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3fa4f6-dbb4-4323-b1ba-dfe7a1dee4c4 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:45.208180 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3fa4f6-dbb4-4323-b1ba-dfe7a1dee4c4 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:45.224004 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:19:45.293712 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3fb1e-3dd2-437f-a2d5-fa8e0f094593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:45.294113 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3fb1e-3dd2-437f-a2d5-fa8e0f094593 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:45.294396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3fb1e-3dd2-437f-a2d5-fa8e0f094593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:45.294706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3fb1e-3dd2-437f-a2d5-fa8e0f094593 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:45.295458 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3fb1e-3dd2-437f-a2d5-fa8e0f094593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:45.297097 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3fb1e-3dd2-437f-a2d5-fa8e0f094593 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:45.308517 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1369/2709] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:19:45.313689 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bcf50-3bb2-42d1-9460-41e3a62f0774 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:45.333445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bcf50-3bb2-42d1-9460-41e3a62f0774 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:45.339029 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa3345-14ae-4821-b3ed-46b9e0a8009f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:45.339960 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa3345-14ae-4821-b3ed-46b9e0a8009f admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:45.340246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa3345-14ae-4821-b3ed-46b9e0a8009f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:45.340526 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa3345-14ae-4821-b3ed-46b9e0a8009f admin admin] PATH_INFO: `/v3/projects/12d69bfb660f48efb90d7fff9bea45d1/users/62c33a95fb124fb78697cfed4f1fe976/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:45.344822 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffa3345-14ae-4821-b3ed-46b9e0a8009f admin admin] RBAC: Authorizing `identity:create_grant(project_id=12d69bfb660f48efb90d7fff9bea45d1, user_id=62c33a95fb124fb78697cfed4f1fe976, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:45.347595 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffa3345-14ae-4821-b3ed-46b9e0a8009f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:45.366404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1341/2710] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:19:45 2022] PUT /identity/v3/projects/12d69bfb660f48efb90d7fff9bea45d1/users/62c33a95fb124fb78697cfed4f1fe976/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:19:51.350679 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361355ec-756f-4d55-a2e5-7cbcd7daaf64 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:51.354833 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361355ec-756f-4d55-a2e5-7cbcd7daaf64 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:51.355612 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361355ec-756f-4d55-a2e5-7cbcd7daaf64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:51.420088 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-361355ec-756f-4d55-a2e5-7cbcd7daaf64 None None] MFA Rules not processed for user `62c33a95fb124fb78697cfed4f1fe976`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:19:51.421657 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-361355ec-756f-4d55-a2e5-7cbcd7daaf64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:51.618779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1370/2711] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:19:51 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:19:51.628898 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa3345-14ae-4821-b3ed-46b9e0a8009f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:51.640319 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361355ec-756f-4d55-a2e5-7cbcd7daaf64 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:51.747365 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa3345-14ae-4821-b3ed-46b9e0a8009f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:51.755872 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304c0f1-74cf-490c-94ab-6f59d6b11179 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:51.756912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304c0f1-74cf-490c-94ab-6f59d6b11179 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:51.757378 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304c0f1-74cf-490c-94ab-6f59d6b11179 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:51.757814 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304c0f1-74cf-490c-94ab-6f59d6b11179 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:51.764805 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361355ec-756f-4d55-a2e5-7cbcd7daaf64 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:51.774801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01218fa0-77be-431c-ba2d-60b0b2d3281e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:51.776046 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01218fa0-77be-431c-ba2d-60b0b2d3281e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:51.776201 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01218fa0-77be-431c-ba2d-60b0b2d3281e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:51.776405 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01218fa0-77be-431c-ba2d-60b0b2d3281e admin admin] PATH_INFO: `/v3/users/614d54ac37d743eea29632bf088e5e27` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:51.778054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01218fa0-77be-431c-ba2d-60b0b2d3281e admin admin] RBAC: Authorizing `identity:delete_user(user_id=614d54ac37d743eea29632bf088e5e27)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:51.780880 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01218fa0-77be-431c-ba2d-60b0b2d3281e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:51.846490 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:19:51.851167 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:19:51.871690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304c0f1-74cf-490c-94ab-6f59d6b11179 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:51.873142 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-01218fa0-77be-431c-ba2d-60b0b2d3281e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '614d54ac37d743eea29632bf088e5e27'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:51.873326 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304c0f1-74cf-490c-94ab-6f59d6b11179 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:51.887132 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-01218fa0-77be-431c-ba2d-60b0b2d3281e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '614d54ac37d743eea29632bf088e5e27'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:51.887619 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1342/2712] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:19:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:19:51.887873 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-01218fa0-77be-431c-ba2d-60b0b2d3281e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '614d54ac37d743eea29632bf088e5e27'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:51.904157 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-01218fa0-77be-431c-ba2d-60b0b2d3281e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '614d54ac37d743eea29632bf088e5e27'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:51.904924 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-01218fa0-77be-431c-ba2d-60b0b2d3281e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '614d54ac37d743eea29632bf088e5e27'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:51.933731 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1371/2713] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:19:51 2022] DELETE /identity/v3/users/614d54ac37d743eea29632bf088e5e27 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:19:51.939637 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf94187-4ef4-4d82-a031-81e90c7f57b6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:51.940384 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf94187-4ef4-4d82-a031-81e90c7f57b6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:51.941320 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf94187-4ef4-4d82-a031-81e90c7f57b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:51.963577 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-edf94187-4ef4-4d82-a031-81e90c7f57b6 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:19:51.965127 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-edf94187-4ef4-4d82-a031-81e90c7f57b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:52.047218 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1343/2714] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:19:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 15:19:52.057772 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01218fa0-77be-431c-ba2d-60b0b2d3281e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:52.059258 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-01218fa0-77be-431c-ba2d-60b0b2d3281e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:52.183617 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01218fa0-77be-431c-ba2d-60b0b2d3281e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:52.189884 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e1170-568e-43ea-b450-97b294b31241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:52.190867 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e1170-568e-43ea-b450-97b294b31241 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:52.191335 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e1170-568e-43ea-b450-97b294b31241 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:52.191788 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e1170-568e-43ea-b450-97b294b31241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:52.218843 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e1170-568e-43ea-b450-97b294b31241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:52.220692 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e1170-568e-43ea-b450-97b294b31241 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:52.236110 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1372/2715] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:19:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:19:52.588321 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf94187-4ef4-4d82-a031-81e90c7f57b6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:52.590739 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-edf94187-4ef4-4d82-a031-81e90c7f57b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:52.615357 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf94187-4ef4-4d82-a031-81e90c7f57b6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:52.622202 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4c5a3-7604-487c-ab46-1dc6cb470775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:52.623239 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4c5a3-7604-487c-ab46-1dc6cb470775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:52.623593 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4c5a3-7604-487c-ab46-1dc6cb470775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:52.623941 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4c5a3-7604-487c-ab46-1dc6cb470775 admin admin] PATH_INFO: `/v3/projects/a3fca267681440049f30d28fe74370b7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:52.626326 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4c5a3-7604-487c-ab46-1dc6cb470775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3fca267681440049f30d28fe74370b7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:52.628870 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4c5a3-7604-487c-ab46-1dc6cb470775 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:52.702969 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-58d4c5a3-7604-487c-ab46-1dc6cb470775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3fca267681440049f30d28fe74370b7', 'request_id': 'req-58d4c5a3-7604-487c-ab46-1dc6cb470775'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:52.724704 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-58d4c5a3-7604-487c-ab46-1dc6cb470775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3fca267681440049f30d28fe74370b7', 'request_id': 'req-58d4c5a3-7604-487c-ab46-1dc6cb470775'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:52.738619 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-58d4c5a3-7604-487c-ab46-1dc6cb470775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3fca267681440049f30d28fe74370b7', 'request_id': 'req-58d4c5a3-7604-487c-ab46-1dc6cb470775'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:52.758261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-58d4c5a3-7604-487c-ab46-1dc6cb470775 admin admin] The token cache is being invalidate because project a3fca267681440049f30d28fe74370b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:19:52.759030 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-58d4c5a3-7604-487c-ab46-1dc6cb470775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:52.760231 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-58d4c5a3-7604-487c-ab46-1dc6cb470775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:52.762326 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1344/2716] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:19:52 2022] DELETE /identity/v3/projects/a3fca267681440049f30d28fe74370b7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 15:19:52.767791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bfce2-d628-447e-961c-ec1d31288674 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:52.768369 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bfce2-d628-447e-961c-ec1d31288674 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:52.768458 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bfce2-d628-447e-961c-ec1d31288674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:52.791653 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-948bfce2-d628-447e-961c-ec1d31288674 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:19:52.793211 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-948bfce2-d628-447e-961c-ec1d31288674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:52.926474 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1373/2717] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:19:52.929085 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42421b-4b3d-4a76-9421-8758e5c0cfe5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:52.929501 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42421b-4b3d-4a76-9421-8758e5c0cfe5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:52.929908 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42421b-4b3d-4a76-9421-8758e5c0cfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:52.977772 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9c42421b-4b3d-4a76-9421-8758e5c0cfe5 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:19:52.980971 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9c42421b-4b3d-4a76-9421-8758e5c0cfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:53.006805 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1345/2718] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:19:53.012229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bfce2-d628-447e-961c-ec1d31288674 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:53.036504 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bfce2-d628-447e-961c-ec1d31288674 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:53.050508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d9ff5-2d6e-499b-9232-754e5a9d56dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:53.053949 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d9ff5-2d6e-499b-9232-754e5a9d56dd admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:53.055572 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d9ff5-2d6e-499b-9232-754e5a9d56dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:53.056129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d9ff5-2d6e-499b-9232-754e5a9d56dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:53.057729 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d9ff5-2d6e-499b-9232-754e5a9d56dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 15:19:53.058383 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d9ff5-2d6e-499b-9232-754e5a9d56dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:53.062598 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d9ff5-2d6e-499b-9232-754e5a9d56dd admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:53.091096 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1374/2719] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 15:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 15:19:53.096791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c42421b-4b3d-4a76-9421-8758e5c0cfe5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:53.115601 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c42421b-4b3d-4a76-9421-8758e5c0cfe5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:53.137439 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1079a43-c9fa-4464-9e48-95526ac630ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:53.138735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1079a43-c9fa-4464-9e48-95526ac630ce admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:53.139326 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1079a43-c9fa-4464-9e48-95526ac630ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:53.139916 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1079a43-c9fa-4464-9e48-95526ac630ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:53.141203 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1079a43-c9fa-4464-9e48-95526ac630ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:53.145348 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1079a43-c9fa-4464-9e48-95526ac630ce admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:53.208556 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1346/2720] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 15:19:53 2022] POST /identity/v3/projects => generated 401 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 15:19:53.215054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d9ff5-2d6e-499b-9232-754e5a9d56dd admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:53.230623 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d9ff5-2d6e-499b-9232-754e5a9d56dd admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:53.288988 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6454bf4-0422-430a-a7b7-f391f4189249 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:53.289549 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6454bf4-0422-430a-a7b7-f391f4189249 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:53.289987 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6454bf4-0422-430a-a7b7-f391f4189249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:53.302490 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:19:53.311743 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b6454bf4-0422-430a-a7b7-f391f4189249 None None] MFA Rules not processed for user `6a8c202f3b4a424cb6565fc8a6abd15b`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:19:53.313456 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b6454bf4-0422-430a-a7b7-f391f4189249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:53.328715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aaae25-dcdb-4185-9aff-6edcf8f47774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:53.329288 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aaae25-dcdb-4185-9aff-6edcf8f47774 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:53.329716 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aaae25-dcdb-4185-9aff-6edcf8f47774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:53.330155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aaae25-dcdb-4185-9aff-6edcf8f47774 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:53.331017 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aaae25-dcdb-4185-9aff-6edcf8f47774 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:53.333088 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aaae25-dcdb-4185-9aff-6edcf8f47774 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:53.362271 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1375/2721] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 15:19:53 2022] POST /identity/v3/users => generated 301 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:19:53.364189 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aaae25-dcdb-4185-9aff-6edcf8f47774 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:53.371638 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:19:53.378385 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aaae25-dcdb-4185-9aff-6edcf8f47774 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:53.386687 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfacbe3-6bff-4c44-9254-4471112f70dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:53.387528 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfacbe3-6bff-4c44-9254-4471112f70dc admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:53.387901 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfacbe3-6bff-4c44-9254-4471112f70dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:53.388316 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfacbe3-6bff-4c44-9254-4471112f70dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:53.389307 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfacbe3-6bff-4c44-9254-4471112f70dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:53.391922 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfacbe3-6bff-4c44-9254-4471112f70dc admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:53.392873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1347/2722] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:19:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 15:19:53.405959 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1376/2723] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:19:53.410639 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6454bf4-0422-430a-a7b7-f391f4189249 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:53.428189 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6454bf4-0422-430a-a7b7-f391f4189249 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:53.434155 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f342f-214d-4fa6-a98c-90be5a16f2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:53.434905 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f342f-214d-4fa6-a98c-90be5a16f2ef admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:53.435192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f342f-214d-4fa6-a98c-90be5a16f2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:53.435468 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f342f-214d-4fa6-a98c-90be5a16f2ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:53.436211 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f342f-214d-4fa6-a98c-90be5a16f2ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:53.442141 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f342f-214d-4fa6-a98c-90be5a16f2ef admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:53.462730 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1348/2724] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:19:53.467629 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfacbe3-6bff-4c44-9254-4471112f70dc admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:53.486177 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfacbe3-6bff-4c44-9254-4471112f70dc admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:53.497292 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a219e-43bf-44e4-bc78-2f2d97835420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:53.499228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a219e-43bf-44e4-bc78-2f2d97835420 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:53.499834 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a219e-43bf-44e4-bc78-2f2d97835420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:53.500417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a219e-43bf-44e4-bc78-2f2d97835420 admin admin] PATH_INFO: `/v3/projects/468f58b95dc24299adca105b5b0ffdc3/users/e0ec0dd1acfb413aaf3e0c5a05805092/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:53.535477 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a219e-43bf-44e4-bc78-2f2d97835420 admin admin] RBAC: Authorizing `identity:create_grant(project_id=468f58b95dc24299adca105b5b0ffdc3, user_id=e0ec0dd1acfb413aaf3e0c5a05805092, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:53.538787 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a219e-43bf-44e4-bc78-2f2d97835420 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:53.556618 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1377/2725] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:19:53 2022] PUT /identity/v3/projects/468f58b95dc24299adca105b5b0ffdc3/users/e0ec0dd1acfb413aaf3e0c5a05805092/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:19:53.563713 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bde185-bf5b-4ec3-a6de-a2b80d6602e5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:53.565239 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bde185-bf5b-4ec3-a6de-a2b80d6602e5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:53.566362 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bde185-bf5b-4ec3-a6de-a2b80d6602e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:53.591570 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-07bde185-bf5b-4ec3-a6de-a2b80d6602e5 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:19:53.593793 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-07bde185-bf5b-4ec3-a6de-a2b80d6602e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:53.731766 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1349/2726] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:19:53.742628 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a219e-43bf-44e4-bc78-2f2d97835420 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:53.746041 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-341a219e-43bf-44e4-bc78-2f2d97835420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:53.864392 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a219e-43bf-44e4-bc78-2f2d97835420 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:53.870211 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c455f3-88a4-4386-92a2-4236d3892706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:53.871213 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c455f3-88a4-4386-92a2-4236d3892706 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:53.871776 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c455f3-88a4-4386-92a2-4236d3892706 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:53.871941 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c455f3-88a4-4386-92a2-4236d3892706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:53.889271 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c455f3-88a4-4386-92a2-4236d3892706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:53.891097 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c455f3-88a4-4386-92a2-4236d3892706 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:53.913561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1378/2727] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:19:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:19:54.784155 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce937d40-88f9-4785-9275-f1d54ac23d97 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:54.784721 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce937d40-88f9-4785-9275-f1d54ac23d97 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:54.784810 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce937d40-88f9-4785-9275-f1d54ac23d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:54.819507 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ce937d40-88f9-4785-9275-f1d54ac23d97 None None] MFA Rules not processed for user `e0ec0dd1acfb413aaf3e0c5a05805092`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:19:54.820867 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ce937d40-88f9-4785-9275-f1d54ac23d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:54.923051 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1350/2728] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:19:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:19:54.925803 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c455f3-88a4-4386-92a2-4236d3892706 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:54.955888 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c455f3-88a4-4386-92a2-4236d3892706 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:54.965315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13bddc-5424-4799-bdbe-9f5cc288acc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:54.966741 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13bddc-5424-4799-bdbe-9f5cc288acc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:54.967505 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13bddc-5424-4799-bdbe-9f5cc288acc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:54.967914 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13bddc-5424-4799-bdbe-9f5cc288acc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:54.969361 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13bddc-5424-4799-bdbe-9f5cc288acc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:54.973475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13bddc-5424-4799-bdbe-9f5cc288acc9 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:55.030705 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1379/2729] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 15:19:54 2022] POST /identity/v3/projects => generated 403 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:19:55.038079 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce937d40-88f9-4785-9275-f1d54ac23d97 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:55.064486 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce937d40-88f9-4785-9275-f1d54ac23d97 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:55.194595 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86161e-425d-4d06-851c-edaf05e1e1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:55.195710 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86161e-425d-4d06-851c-edaf05e1e1a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:55.196075 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86161e-425d-4d06-851c-edaf05e1e1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:55.196450 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86161e-425d-4d06-851c-edaf05e1e1a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:55.198059 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86161e-425d-4d06-851c-edaf05e1e1a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:55.200269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86161e-425d-4d06-851c-edaf05e1e1a1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:55.216211 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:19:55.234726 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1351/2730] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 15:19:55 2022] POST /identity/v3/users => generated 308 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:19:55.236217 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13bddc-5424-4799-bdbe-9f5cc288acc9 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:55.249545 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13bddc-5424-4799-bdbe-9f5cc288acc9 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:55.255353 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7662e0d6-a796-4c35-b51d-2ab28e26b8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:55.256154 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662e0d6-a796-4c35-b51d-2ab28e26b8c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:55.256507 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662e0d6-a796-4c35-b51d-2ab28e26b8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:55.257011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662e0d6-a796-4c35-b51d-2ab28e26b8c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:55.257956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662e0d6-a796-4c35-b51d-2ab28e26b8c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:55.260434 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662e0d6-a796-4c35-b51d-2ab28e26b8c7 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:55.279730 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1380/2731] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 15:19:55.281873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86161e-425d-4d06-851c-edaf05e1e1a1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:55.301417 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86161e-425d-4d06-851c-edaf05e1e1a1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:55.310792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4acb56-d3e2-4c78-83de-5cbab75e5e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:55.312250 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4acb56-d3e2-4c78-83de-5cbab75e5e75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:55.312649 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4acb56-d3e2-4c78-83de-5cbab75e5e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:55.313086 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4acb56-d3e2-4c78-83de-5cbab75e5e75 admin admin] PATH_INFO: `/v3/projects/fab53c2947ec4d3786854c84cab468a9/users/9e564edec0494c58b8cf6b9e260cd285/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:55.343088 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4acb56-d3e2-4c78-83de-5cbab75e5e75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fab53c2947ec4d3786854c84cab468a9, user_id=9e564edec0494c58b8cf6b9e260cd285, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:55.349148 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4acb56-d3e2-4c78-83de-5cbab75e5e75 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:55.372575 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1352/2732] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:19:55 2022] PUT /identity/v3/projects/fab53c2947ec4d3786854c84cab468a9/users/9e564edec0494c58b8cf6b9e260cd285/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:19:55.377507 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7662e0d6-a796-4c35-b51d-2ab28e26b8c7 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:55.391487 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7662e0d6-a796-4c35-b51d-2ab28e26b8c7 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:55.473566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1d026-a6ec-4c74-9885-fb05583e388e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:55.474124 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1d026-a6ec-4c74-9885-fb05583e388e admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:55.474415 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1d026-a6ec-4c74-9885-fb05583e388e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:55.474729 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1d026-a6ec-4c74-9885-fb05583e388e admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:55.475486 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a1d026-a6ec-4c74-9885-fb05583e388e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:55.477172 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a1d026-a6ec-4c74-9885-fb05583e388e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:55.489286 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1381/2733] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:19:55.494416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4acb56-d3e2-4c78-83de-5cbab75e5e75 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:55.506691 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4acb56-d3e2-4c78-83de-5cbab75e5e75 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:55.513541 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbe843-157b-4582-a02d-4dbc6013a040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:55.514464 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbe843-157b-4582-a02d-4dbc6013a040 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:55.514929 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbe843-157b-4582-a02d-4dbc6013a040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:55.515238 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbe843-157b-4582-a02d-4dbc6013a040 admin admin] PATH_INFO: `/v3/projects/fab53c2947ec4d3786854c84cab468a9/users/9e564edec0494c58b8cf6b9e260cd285/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:55.517513 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbe843-157b-4582-a02d-4dbc6013a040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fab53c2947ec4d3786854c84cab468a9, user_id=9e564edec0494c58b8cf6b9e260cd285, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:55.520531 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbe843-157b-4582-a02d-4dbc6013a040 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:55.533079 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1353/2734] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:19:55 2022] PUT /identity/v3/projects/fab53c2947ec4d3786854c84cab468a9/users/9e564edec0494c58b8cf6b9e260cd285/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 15:19:56.633356 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873db9e-6dfe-40da-b519-15f9fabe9560 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:56.634268 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873db9e-6dfe-40da-b519-15f9fabe9560 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:56.634810 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873db9e-6dfe-40da-b519-15f9fabe9560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:56.641328 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:19:56.674499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-f873db9e-6dfe-40da-b519-15f9fabe9560 None None] MFA Rules not processed for user `9e564edec0494c58b8cf6b9e260cd285`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:19:56.676363 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f873db9e-6dfe-40da-b519-15f9fabe9560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:56.794084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1382/2735] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:19:56 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 15:19:58.100968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbe843-157b-4582-a02d-4dbc6013a040 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:58.196193 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbe843-157b-4582-a02d-4dbc6013a040 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:58.201355 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07814a-9d8d-455c-86c2-1a6bbcea22fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:58.202268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07814a-9d8d-455c-86c2-1a6bbcea22fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:58.202744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07814a-9d8d-455c-86c2-1a6bbcea22fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:58.203032 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07814a-9d8d-455c-86c2-1a6bbcea22fd admin admin] PATH_INFO: `/v3/users/e0ec0dd1acfb413aaf3e0c5a05805092` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:58.204290 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07814a-9d8d-455c-86c2-1a6bbcea22fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0ec0dd1acfb413aaf3e0c5a05805092)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:58.206310 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07814a-9d8d-455c-86c2-1a6bbcea22fd admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:58.272729 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0e07814a-9d8d-455c-86c2-1a6bbcea22fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0ec0dd1acfb413aaf3e0c5a05805092'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:58.286337 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0e07814a-9d8d-455c-86c2-1a6bbcea22fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0ec0dd1acfb413aaf3e0c5a05805092'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:58.295558 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0e07814a-9d8d-455c-86c2-1a6bbcea22fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0ec0dd1acfb413aaf3e0c5a05805092'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:58.309164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0e07814a-9d8d-455c-86c2-1a6bbcea22fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0ec0dd1acfb413aaf3e0c5a05805092'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:58.309920 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0e07814a-9d8d-455c-86c2-1a6bbcea22fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0ec0dd1acfb413aaf3e0c5a05805092'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:58.316743 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1354/2736] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:19:58 2022] DELETE /identity/v3/users/e0ec0dd1acfb413aaf3e0c5a05805092 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:19:58.502164 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873db9e-6dfe-40da-b519-15f9fabe9560 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:58.518065 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873db9e-6dfe-40da-b519-15f9fabe9560 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:58.581749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c681b1-0a2d-4d5f-ba38-5a497ee323e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:58.582679 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c681b1-0a2d-4d5f-ba38-5a497ee323e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:58.582975 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c681b1-0a2d-4d5f-ba38-5a497ee323e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:58.583258 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c681b1-0a2d-4d5f-ba38-5a497ee323e5 admin admin] PATH_INFO: `/v3/projects/468f58b95dc24299adca105b5b0ffdc3` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:58.584482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c681b1-0a2d-4d5f-ba38-5a497ee323e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=468f58b95dc24299adca105b5b0ffdc3)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:58.586403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c681b1-0a2d-4d5f-ba38-5a497ee323e5 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:58.637107 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-78c681b1-0a2d-4d5f-ba38-5a497ee323e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '468f58b95dc24299adca105b5b0ffdc3', 'request_id': 'req-78c681b1-0a2d-4d5f-ba38-5a497ee323e5'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:58.642800 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-78c681b1-0a2d-4d5f-ba38-5a497ee323e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '468f58b95dc24299adca105b5b0ffdc3', 'request_id': 'req-78c681b1-0a2d-4d5f-ba38-5a497ee323e5'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:58.649423 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-78c681b1-0a2d-4d5f-ba38-5a497ee323e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '468f58b95dc24299adca105b5b0ffdc3', 'request_id': 'req-78c681b1-0a2d-4d5f-ba38-5a497ee323e5'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:58.664925 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-78c681b1-0a2d-4d5f-ba38-5a497ee323e5 admin admin] The token cache is being invalidate because project 468f58b95dc24299adca105b5b0ffdc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:19:58.665212 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-78c681b1-0a2d-4d5f-ba38-5a497ee323e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:58.665889 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-78c681b1-0a2d-4d5f-ba38-5a497ee323e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:58.667546 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1383/2737] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:19:58 2022] DELETE /identity/v3/projects/468f58b95dc24299adca105b5b0ffdc3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:19:58.671894 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07814a-9d8d-455c-86c2-1a6bbcea22fd admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:58.673485 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0e07814a-9d8d-455c-86c2-1a6bbcea22fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:58.687794 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07814a-9d8d-455c-86c2-1a6bbcea22fd admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:58.746142 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:19:58.786295 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d29ac-60c5-4cfa-a7a5-9cc0c7d20cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:58.787282 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d29ac-60c5-4cfa-a7a5-9cc0c7d20cd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:58.787571 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d29ac-60c5-4cfa-a7a5-9cc0c7d20cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:58.787914 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d29ac-60c5-4cfa-a7a5-9cc0c7d20cd6 admin admin] PATH_INFO: `/v3/users/9e564edec0494c58b8cf6b9e260cd285` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:58.789062 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d29ac-60c5-4cfa-a7a5-9cc0c7d20cd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e564edec0494c58b8cf6b9e260cd285)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:58.791039 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d29ac-60c5-4cfa-a7a5-9cc0c7d20cd6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:58.850882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-be2d29ac-60c5-4cfa-a7a5-9cc0c7d20cd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e564edec0494c58b8cf6b9e260cd285'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:58.859630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-be2d29ac-60c5-4cfa-a7a5-9cc0c7d20cd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e564edec0494c58b8cf6b9e260cd285'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:58.867323 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-be2d29ac-60c5-4cfa-a7a5-9cc0c7d20cd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e564edec0494c58b8cf6b9e260cd285'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:58.879856 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-be2d29ac-60c5-4cfa-a7a5-9cc0c7d20cd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e564edec0494c58b8cf6b9e260cd285'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:58.880625 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-be2d29ac-60c5-4cfa-a7a5-9cc0c7d20cd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e564edec0494c58b8cf6b9e260cd285'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:58.888729 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1355/2738] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:19:58 2022] DELETE /identity/v3/users/9e564edec0494c58b8cf6b9e260cd285 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:19:59.093834 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c681b1-0a2d-4d5f-ba38-5a497ee323e5 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:59.095609 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-78c681b1-0a2d-4d5f-ba38-5a497ee323e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:59.115058 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c681b1-0a2d-4d5f-ba38-5a497ee323e5 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:59.134968 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:19:59.189650 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbd67f-79c2-4bed-b323-785784a06482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:59.190550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbd67f-79c2-4bed-b323-785784a06482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:59.190875 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbd67f-79c2-4bed-b323-785784a06482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:59.191157 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbd67f-79c2-4bed-b323-785784a06482 admin admin] PATH_INFO: `/v3/projects/fab53c2947ec4d3786854c84cab468a9` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:59.192413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fbd67f-79c2-4bed-b323-785784a06482 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fab53c2947ec4d3786854c84cab468a9)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:59.194564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fbd67f-79c2-4bed-b323-785784a06482 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:59.245605 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-65fbd67f-79c2-4bed-b323-785784a06482 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fab53c2947ec4d3786854c84cab468a9', 'request_id': 'req-65fbd67f-79c2-4bed-b323-785784a06482'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:59.251411 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-65fbd67f-79c2-4bed-b323-785784a06482 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fab53c2947ec4d3786854c84cab468a9', 'request_id': 'req-65fbd67f-79c2-4bed-b323-785784a06482'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:59.257693 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-65fbd67f-79c2-4bed-b323-785784a06482 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fab53c2947ec4d3786854c84cab468a9', 'request_id': 'req-65fbd67f-79c2-4bed-b323-785784a06482'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:59.271784 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-65fbd67f-79c2-4bed-b323-785784a06482 admin admin] The token cache is being invalidate because project fab53c2947ec4d3786854c84cab468a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:19:59.272078 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-65fbd67f-79c2-4bed-b323-785784a06482 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:59.272740 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-65fbd67f-79c2-4bed-b323-785784a06482 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:19:59.274919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1384/2739] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:19:59 2022] DELETE /identity/v3/projects/fab53c2947ec4d3786854c84cab468a9 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 15:19:59.280378 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c1b64-df74-4a34-8386-c6a8e88ea6ad None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:59.281009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c1b64-df74-4a34-8386-c6a8e88ea6ad None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:59.281009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c1b64-df74-4a34-8386-c6a8e88ea6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:59.300971 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1f7c1b64-df74-4a34-8386-c6a8e88ea6ad None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:19:59.302252 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1f7c1b64-df74-4a34-8386-c6a8e88ea6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:59.382324 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1356/2740] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:19:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 15:19:59.390921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5143d5-0a74-4444-ae17-e69d7e04b819 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:59.391214 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5143d5-0a74-4444-ae17-e69d7e04b819 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:59.391498 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5143d5-0a74-4444-ae17-e69d7e04b819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:59.413637 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6c5143d5-0a74-4444-ae17-e69d7e04b819 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:19:59.414938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6c5143d5-0a74-4444-ae17-e69d7e04b819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:59.424017 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1385/2741] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:19:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:19:59.429065 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c1b64-df74-4a34-8386-c6a8e88ea6ad None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:59.439040 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c1b64-df74-4a34-8386-c6a8e88ea6ad None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:59.443779 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc436c-9d1c-488d-83f8-0b299dd44638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:59.444467 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc436c-9d1c-488d-83f8-0b299dd44638 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:59.444751 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc436c-9d1c-488d-83f8-0b299dd44638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:59.445031 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc436c-9d1c-488d-83f8-0b299dd44638 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:59.445764 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc436c-9d1c-488d-83f8-0b299dd44638 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 15:19:59.446074 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc436c-9d1c-488d-83f8-0b299dd44638 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:59.448211 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc436c-9d1c-488d-83f8-0b299dd44638 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:59.461139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1357/2742] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 15:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 15:19:59.466269 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5143d5-0a74-4444-ae17-e69d7e04b819 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:59.475831 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5143d5-0a74-4444-ae17-e69d7e04b819 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:59.480672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e48d2-846b-4099-894d-644e3605fe33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:59.481345 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e48d2-846b-4099-894d-644e3605fe33 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:59.481626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e48d2-846b-4099-894d-644e3605fe33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:59.481905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e48d2-846b-4099-894d-644e3605fe33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:59.482609 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e48d2-846b-4099-894d-644e3605fe33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:59.484522 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e48d2-846b-4099-894d-644e3605fe33 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:59.513255 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1386/2743] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 15:19:59 2022] POST /identity/v3/projects => generated 357 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:19:59.517332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc436c-9d1c-488d-83f8-0b299dd44638 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:59.528369 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc436c-9d1c-488d-83f8-0b299dd44638 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:59.573197 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:19:59.613582 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7889df7-3fd9-4c5e-83b6-e1e35a1b7b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:59.614840 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7889df7-3fd9-4c5e-83b6-e1e35a1b7b41 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:59.615343 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7889df7-3fd9-4c5e-83b6-e1e35a1b7b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:59.615842 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7889df7-3fd9-4c5e-83b6-e1e35a1b7b41 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:59.617012 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7889df7-3fd9-4c5e-83b6-e1e35a1b7b41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:59.620986 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7889df7-3fd9-4c5e-83b6-e1e35a1b7b41 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:59.656449 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1358/2744] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 15:19:59 2022] POST /identity/v3/users => generated 279 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:19:59.661324 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e48d2-846b-4099-894d-644e3605fe33 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:59.672209 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e48d2-846b-4099-894d-644e3605fe33 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:59.677435 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9255e8a-df61-491d-8837-59007f87823f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:59.678141 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9255e8a-df61-491d-8837-59007f87823f admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:59.678425 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9255e8a-df61-491d-8837-59007f87823f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:59.678732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9255e8a-df61-491d-8837-59007f87823f admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:59.679481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9255e8a-df61-491d-8837-59007f87823f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:59.681117 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9255e8a-df61-491d-8837-59007f87823f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:59.694952 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1387/2745] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:19:59.699895 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7889df7-3fd9-4c5e-83b6-e1e35a1b7b41 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:59.710822 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7889df7-3fd9-4c5e-83b6-e1e35a1b7b41 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:59.717171 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e09bcf-7f7d-4947-8f89-0f3c4d91fc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:59.717915 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e09bcf-7f7d-4947-8f89-0f3c4d91fc5d admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:59.718209 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e09bcf-7f7d-4947-8f89-0f3c4d91fc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:59.718496 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e09bcf-7f7d-4947-8f89-0f3c4d91fc5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:59.719287 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e09bcf-7f7d-4947-8f89-0f3c4d91fc5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:59.720949 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e09bcf-7f7d-4947-8f89-0f3c4d91fc5d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:59.732770 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1359/2746] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:19:59.737091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9255e8a-df61-491d-8837-59007f87823f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:59.750339 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9255e8a-df61-491d-8837-59007f87823f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:19:59.756148 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7ed8f-60ec-46a1-98d1-90d48da86600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:19:59.757275 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7ed8f-60ec-46a1-98d1-90d48da86600 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:59.757694 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7ed8f-60ec-46a1-98d1-90d48da86600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:59.757980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7ed8f-60ec-46a1-98d1-90d48da86600 admin admin] PATH_INFO: `/v3/projects/f1fb24a08d374c43ab2b35ac45acdf84/users/bc20a048929546baa3125a6c5b8a1351/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:59.786674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7ed8f-60ec-46a1-98d1-90d48da86600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1fb24a08d374c43ab2b35ac45acdf84, user_id=bc20a048929546baa3125a6c5b8a1351, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:19:59.791944 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7ed8f-60ec-46a1-98d1-90d48da86600 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:19:59.813485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1388/2747] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:19:59 2022] PUT /identity/v3/projects/f1fb24a08d374c43ab2b35ac45acdf84/users/bc20a048929546baa3125a6c5b8a1351/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:19:59.824455 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ded80e-2e95-4dda-abd3-9ad931748cb1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:19:59.824882 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ded80e-2e95-4dda-abd3-9ad931748cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:19:59.825253 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ded80e-2e95-4dda-abd3-9ad931748cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:19:59.855068 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-26ded80e-2e95-4dda-abd3-9ad931748cb1 None None] MFA Rules not processed for user `bc20a048929546baa3125a6c5b8a1351`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:19:59.856311 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-26ded80e-2e95-4dda-abd3-9ad931748cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:19:59.935988 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1360/2748] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:19:59 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 15:19:59.940937 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7ed8f-60ec-46a1-98d1-90d48da86600 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:19:59.994495 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:00.015953 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7ed8f-60ec-46a1-98d1-90d48da86600 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:00.021236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9261398-0816-4596-9162-46fda26676b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:00.021954 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9261398-0816-4596-9162-46fda26676b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:00.022240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9261398-0816-4596-9162-46fda26676b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:00.022531 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9261398-0816-4596-9162-46fda26676b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:00.023293 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9261398-0816-4596-9162-46fda26676b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:00.025295 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9261398-0816-4596-9162-46fda26676b9 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:00.054763 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1389/2749] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 15:19:59 2022] POST /identity/v3/projects => generated 357 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:20:00.059469 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ded80e-2e95-4dda-abd3-9ad931748cb1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:00.071217 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ded80e-2e95-4dda-abd3-9ad931748cb1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:00.137633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d0913-22e9-4c92-88d9-fcbad34012ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:00.138323 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d0913-22e9-4c92-88d9-fcbad34012ab admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:00.138611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d0913-22e9-4c92-88d9-fcbad34012ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:00.138947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d0913-22e9-4c92-88d9-fcbad34012ab admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:00.139785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d0913-22e9-4c92-88d9-fcbad34012ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:00.141743 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d0913-22e9-4c92-88d9-fcbad34012ab admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:00.168648 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1361/2750] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 15:20:00 2022] POST /identity/v3/users => generated 285 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 15:20:00.169917 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9261398-0816-4596-9162-46fda26676b9 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:00.181182 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9261398-0816-4596-9162-46fda26676b9 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:00.187208 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04218cb5-7c96-4296-8cc9-335606a3233e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:00.187897 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04218cb5-7c96-4296-8cc9-335606a3233e admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:00.188183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04218cb5-7c96-4296-8cc9-335606a3233e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:00.188460 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04218cb5-7c96-4296-8cc9-335606a3233e admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:00.189200 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04218cb5-7c96-4296-8cc9-335606a3233e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:00.190854 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04218cb5-7c96-4296-8cc9-335606a3233e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:00.203216 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1390/2751] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 15:20:00.208628 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d0913-22e9-4c92-88d9-fcbad34012ab admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:00.219899 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d0913-22e9-4c92-88d9-fcbad34012ab admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:00.224880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84286f26-2e62-48bf-9d78-13cee5547633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:00.225810 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84286f26-2e62-48bf-9d78-13cee5547633 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:00.226097 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84286f26-2e62-48bf-9d78-13cee5547633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:00.226376 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84286f26-2e62-48bf-9d78-13cee5547633 admin admin] PATH_INFO: `/v3/projects/bb025fd3d9d440e0ada89eace839e038/users/15723c8fc0c54729bf3eb16d38dd9206/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:00.245499 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84286f26-2e62-48bf-9d78-13cee5547633 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb025fd3d9d440e0ada89eace839e038, user_id=15723c8fc0c54729bf3eb16d38dd9206, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:00.248213 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84286f26-2e62-48bf-9d78-13cee5547633 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:00.264372 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1362/2752] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:20:00 2022] PUT /identity/v3/projects/bb025fd3d9d440e0ada89eace839e038/users/15723c8fc0c54729bf3eb16d38dd9206/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:00.268633 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04218cb5-7c96-4296-8cc9-335606a3233e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:00.279409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04218cb5-7c96-4296-8cc9-335606a3233e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:00.345555 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcc976-a3ae-4c62-ba66-2eb589c00571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:00.346288 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcc976-a3ae-4c62-ba66-2eb589c00571 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:00.346574 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcc976-a3ae-4c62-ba66-2eb589c00571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:00.346903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcc976-a3ae-4c62-ba66-2eb589c00571 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:00.354317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dcc976-a3ae-4c62-ba66-2eb589c00571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:00.354317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dcc976-a3ae-4c62-ba66-2eb589c00571 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:00.370581 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1391/2753] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 15:20:00.375456 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84286f26-2e62-48bf-9d78-13cee5547633 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:00.388171 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84286f26-2e62-48bf-9d78-13cee5547633 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:00.393083 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2dd3bd-bfdf-4102-9df9-b23bf203cd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:00.394219 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2dd3bd-bfdf-4102-9df9-b23bf203cd2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:00.394564 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2dd3bd-bfdf-4102-9df9-b23bf203cd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:00.394969 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2dd3bd-bfdf-4102-9df9-b23bf203cd2c admin admin] PATH_INFO: `/v3/projects/bb025fd3d9d440e0ada89eace839e038/users/15723c8fc0c54729bf3eb16d38dd9206/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:00.403866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2dd3bd-bfdf-4102-9df9-b23bf203cd2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb025fd3d9d440e0ada89eace839e038, user_id=15723c8fc0c54729bf3eb16d38dd9206, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:00.407092 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2dd3bd-bfdf-4102-9df9-b23bf203cd2c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:00.420689 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1363/2754] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:20:00 2022] PUT /identity/v3/projects/bb025fd3d9d440e0ada89eace839e038/users/15723c8fc0c54729bf3eb16d38dd9206/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:00.433923 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d8e5b-0508-4c18-bcbd-be854b0cde49 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:00.434240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d8e5b-0508-4c18-bcbd-be854b0cde49 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:00.434638 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d8e5b-0508-4c18-bcbd-be854b0cde49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:00.470121 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-565d8e5b-0508-4c18-bcbd-be854b0cde49 None None] MFA Rules not processed for user `15723c8fc0c54729bf3eb16d38dd9206`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:00.471461 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-565d8e5b-0508-4c18-bcbd-be854b0cde49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:00.553283 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1392/2755] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:00 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:20:00.565409 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2dd3bd-bfdf-4102-9df9-b23bf203cd2c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:00.567165 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6f2dd3bd-bfdf-4102-9df9-b23bf203cd2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:00.573834 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:00.636432 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2dd3bd-bfdf-4102-9df9-b23bf203cd2c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:00.640220 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0574fa-c752-4502-8082-977f624fe8e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:00.640890 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0574fa-c752-4502-8082-977f624fe8e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:00.641178 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0574fa-c752-4502-8082-977f624fe8e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:00.641456 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0574fa-c752-4502-8082-977f624fe8e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:00.652276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0574fa-c752-4502-8082-977f624fe8e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:00.653733 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0574fa-c752-4502-8082-977f624fe8e3 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:00.664370 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1364/2756] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:20:00 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:20:00.754066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d8e5b-0508-4c18-bcbd-be854b0cde49 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:00.763992 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d8e5b-0508-4c18-bcbd-be854b0cde49 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:00.767868 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d427645-f076-461c-ad26-8ea06d670dd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:00.768460 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d427645-f076-461c-ad26-8ea06d670dd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:00.768877 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d427645-f076-461c-ad26-8ea06d670dd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:00.769087 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d427645-f076-461c-ad26-8ea06d670dd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:00.820064 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:00.846850 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d427645-f076-461c-ad26-8ea06d670dd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:00.848321 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d427645-f076-461c-ad26-8ea06d670dd0 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:00.861813 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1393/2757] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:20:00 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:20:10.659619 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=24 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:10.853626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=10 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:15.139829 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0574fa-c752-4502-8082-977f624fe8e3 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:15.141487 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7d0574fa-c752-4502-8082-977f624fe8e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:15.235522 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0574fa-c752-4502-8082-977f624fe8e3 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:15.241709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0cc03-2e6e-4827-a67a-7aa8d23c2d21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:15.242735 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0cc03-2e6e-4827-a67a-7aa8d23c2d21 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:15.243220 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0cc03-2e6e-4827-a67a-7aa8d23c2d21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:15.243598 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0cc03-2e6e-4827-a67a-7aa8d23c2d21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:15.246010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7cb0cc03-2e6e-4827-a67a-7aa8d23c2d21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:15.265036 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb0cc03-2e6e-4827-a67a-7aa8d23c2d21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:15.266492 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb0cc03-2e6e-4827-a67a-7aa8d23c2d21 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:15.276628 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1365/2758] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:20:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:20:18.521977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d427645-f076-461c-ad26-8ea06d670dd0 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:18.590574 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d427645-f076-461c-ad26-8ea06d670dd0 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:18.595334 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4118dab-6aeb-4367-b4c4-eb741f11e303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:18.596359 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4118dab-6aeb-4367-b4c4-eb741f11e303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:18.596634 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4118dab-6aeb-4367-b4c4-eb741f11e303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:18.596810 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4118dab-6aeb-4367-b4c4-eb741f11e303 admin admin] PATH_INFO: `/v3/users/bc20a048929546baa3125a6c5b8a1351` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:18.598220 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4118dab-6aeb-4367-b4c4-eb741f11e303 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc20a048929546baa3125a6c5b8a1351)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:18.600236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4118dab-6aeb-4367-b4c4-eb741f11e303 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:18.659470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f4118dab-6aeb-4367-b4c4-eb741f11e303 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc20a048929546baa3125a6c5b8a1351'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:18.673950 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f4118dab-6aeb-4367-b4c4-eb741f11e303 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc20a048929546baa3125a6c5b8a1351'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:18.674758 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f4118dab-6aeb-4367-b4c4-eb741f11e303 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc20a048929546baa3125a6c5b8a1351'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:18.693221 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f4118dab-6aeb-4367-b4c4-eb741f11e303 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc20a048929546baa3125a6c5b8a1351'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:18.694810 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f4118dab-6aeb-4367-b4c4-eb741f11e303 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc20a048929546baa3125a6c5b8a1351'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:18.717583 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1394/2759] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:20:18 2022] DELETE /identity/v3/users/bc20a048929546baa3125a6c5b8a1351 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:18.724182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f2ea3-379c-45a4-923c-c695b99e3f10 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:18.724979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f2ea3-379c-45a4-923c-c695b99e3f10 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:18.724979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f2ea3-379c-45a4-923c-c695b99e3f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:18.746232 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7d0f2ea3-379c-45a4-923c-c695b99e3f10 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:18.747598 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7d0f2ea3-379c-45a4-923c-c695b99e3f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:18.817051 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1366/2760] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:20:18.830792 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4118dab-6aeb-4367-b4c4-eb741f11e303 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:18.833519 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f4118dab-6aeb-4367-b4c4-eb741f11e303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:18.912324 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4118dab-6aeb-4367-b4c4-eb741f11e303 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:18.918582 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f553304-eb8a-494b-a10f-246f4cc48c60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:18.919946 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f553304-eb8a-494b-a10f-246f4cc48c60 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:18.920597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f553304-eb8a-494b-a10f-246f4cc48c60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:18.921308 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f553304-eb8a-494b-a10f-246f4cc48c60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:18.940485 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f553304-eb8a-494b-a10f-246f4cc48c60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:18.944373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f553304-eb8a-494b-a10f-246f4cc48c60 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:18.958607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1395/2761] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:20:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:20:19.350152 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f2ea3-379c-45a4-923c-c695b99e3f10 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:19.352618 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7d0f2ea3-379c-45a4-923c-c695b99e3f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:19.379557 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f2ea3-379c-45a4-923c-c695b99e3f10 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:19.390399 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3a4c4-e44c-4815-aede-904408fa07dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:19.405617 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3a4c4-e44c-4815-aede-904408fa07dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:19.406546 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3a4c4-e44c-4815-aede-904408fa07dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:19.410103 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3a4c4-e44c-4815-aede-904408fa07dc admin admin] PATH_INFO: `/v3/projects/f1fb24a08d374c43ab2b35ac45acdf84` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:19.415642 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c3a4c4-e44c-4815-aede-904408fa07dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1fb24a08d374c43ab2b35ac45acdf84)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:19.422392 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c3a4c4-e44c-4815-aede-904408fa07dc admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:19.514575 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54c3a4c4-e44c-4815-aede-904408fa07dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1fb24a08d374c43ab2b35ac45acdf84', 'request_id': 'req-54c3a4c4-e44c-4815-aede-904408fa07dc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:19.522222 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54c3a4c4-e44c-4815-aede-904408fa07dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1fb24a08d374c43ab2b35ac45acdf84', 'request_id': 'req-54c3a4c4-e44c-4815-aede-904408fa07dc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:19.530926 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54c3a4c4-e44c-4815-aede-904408fa07dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1fb24a08d374c43ab2b35ac45acdf84', 'request_id': 'req-54c3a4c4-e44c-4815-aede-904408fa07dc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:19.551201 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54c3a4c4-e44c-4815-aede-904408fa07dc admin admin] The token cache is being invalidate because project f1fb24a08d374c43ab2b35ac45acdf84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:20:19.551611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54c3a4c4-e44c-4815-aede-904408fa07dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:19.553305 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54c3a4c4-e44c-4815-aede-904408fa07dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:19.554281 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1367/2762] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:20:19 2022] DELETE /identity/v3/projects/f1fb24a08d374c43ab2b35ac45acdf84 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:19.558804 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f553304-eb8a-494b-a10f-246f4cc48c60 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:19.576608 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f553304-eb8a-494b-a10f-246f4cc48c60 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:19.622887 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:19.694873 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cbadf-2ff4-4a0f-9926-c321ed37a9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:19.696091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cbadf-2ff4-4a0f-9926-c321ed37a9dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:19.696885 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cbadf-2ff4-4a0f-9926-c321ed37a9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:19.697332 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cbadf-2ff4-4a0f-9926-c321ed37a9dc admin admin] PATH_INFO: `/v3/users/15723c8fc0c54729bf3eb16d38dd9206` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:19.698809 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2cbadf-2ff4-4a0f-9926-c321ed37a9dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=15723c8fc0c54729bf3eb16d38dd9206)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:19.700738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2cbadf-2ff4-4a0f-9926-c321ed37a9dc admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:19.771636 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ce2cbadf-2ff4-4a0f-9926-c321ed37a9dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15723c8fc0c54729bf3eb16d38dd9206'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:19.782906 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ce2cbadf-2ff4-4a0f-9926-c321ed37a9dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15723c8fc0c54729bf3eb16d38dd9206'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:19.784360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ce2cbadf-2ff4-4a0f-9926-c321ed37a9dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15723c8fc0c54729bf3eb16d38dd9206'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:19.809409 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ce2cbadf-2ff4-4a0f-9926-c321ed37a9dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15723c8fc0c54729bf3eb16d38dd9206'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:19.810449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-ce2cbadf-2ff4-4a0f-9926-c321ed37a9dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15723c8fc0c54729bf3eb16d38dd9206'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:19.826091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1396/2763] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:20:19 2022] DELETE /identity/v3/users/15723c8fc0c54729bf3eb16d38dd9206 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:20.197152 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3a4c4-e44c-4815-aede-904408fa07dc admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:20.199188 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-54c3a4c4-e44c-4815-aede-904408fa07dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:20.220633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3a4c4-e44c-4815-aede-904408fa07dc admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:20.240394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:20.305394 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab25f9-8df0-47a5-a166-b6e4727dde5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:20.306513 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab25f9-8df0-47a5-a166-b6e4727dde5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:20.308111 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab25f9-8df0-47a5-a166-b6e4727dde5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:20.308601 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab25f9-8df0-47a5-a166-b6e4727dde5f admin admin] PATH_INFO: `/v3/projects/bb025fd3d9d440e0ada89eace839e038` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:20.315851 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab25f9-8df0-47a5-a166-b6e4727dde5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb025fd3d9d440e0ada89eace839e038)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:20.319478 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab25f9-8df0-47a5-a166-b6e4727dde5f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:20.415401 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-17ab25f9-8df0-47a5-a166-b6e4727dde5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb025fd3d9d440e0ada89eace839e038', 'request_id': 'req-17ab25f9-8df0-47a5-a166-b6e4727dde5f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:20.422677 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-17ab25f9-8df0-47a5-a166-b6e4727dde5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb025fd3d9d440e0ada89eace839e038', 'request_id': 'req-17ab25f9-8df0-47a5-a166-b6e4727dde5f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:20.435285 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-17ab25f9-8df0-47a5-a166-b6e4727dde5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb025fd3d9d440e0ada89eace839e038', 'request_id': 'req-17ab25f9-8df0-47a5-a166-b6e4727dde5f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:20.470866 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-17ab25f9-8df0-47a5-a166-b6e4727dde5f admin admin] The token cache is being invalidate because project bb025fd3d9d440e0ada89eace839e038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:20:20.471094 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-17ab25f9-8df0-47a5-a166-b6e4727dde5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:20.471798 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-17ab25f9-8df0-47a5-a166-b6e4727dde5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:20.475369 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1368/2764] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:20:20 2022] DELETE /identity/v3/projects/bb025fd3d9d440e0ada89eace839e038 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:20.480266 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49afc0b-0eff-4c70-aa24-b6ab72285678 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:20.480550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49afc0b-0eff-4c70-aa24-b6ab72285678 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:20.480832 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49afc0b-0eff-4c70-aa24-b6ab72285678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:20.510912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-b49afc0b-0eff-4c70-aa24-b6ab72285678 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:20.512233 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b49afc0b-0eff-4c70-aa24-b6ab72285678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:20.611452 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1397/2765] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:20:20.620004 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91e48d-0483-46d2-a7a2-2e06f4167a32 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:20.620742 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91e48d-0483-46d2-a7a2-2e06f4167a32 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:20.621376 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91e48d-0483-46d2-a7a2-2e06f4167a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:20.652260 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2f91e48d-0483-46d2-a7a2-2e06f4167a32 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:20.654569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2f91e48d-0483-46d2-a7a2-2e06f4167a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:20.670456 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1369/2766] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:20:20.680154 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49afc0b-0eff-4c70-aa24-b6ab72285678 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:20.692726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49afc0b-0eff-4c70-aa24-b6ab72285678 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:20.699294 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cb19d-207a-463a-a870-999c162f556a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:20.700168 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cb19d-207a-463a-a870-999c162f556a admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:20.700482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cb19d-207a-463a-a870-999c162f556a admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:20.700768 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cb19d-207a-463a-a870-999c162f556a admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:20.701442 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999cb19d-207a-463a-a870-999c162f556a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 15:20:20.701742 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999cb19d-207a-463a-a870-999c162f556a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:20.703583 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999cb19d-207a-463a-a870-999c162f556a admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:20.721673 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1398/2767] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 15:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 15:20:20.726235 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91e48d-0483-46d2-a7a2-2e06f4167a32 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:20.736593 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91e48d-0483-46d2-a7a2-2e06f4167a32 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:20.741952 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad65002-95a4-4d13-b445-5724e2dc4642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:20.742783 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad65002-95a4-4d13-b445-5724e2dc4642 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:20.743590 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad65002-95a4-4d13-b445-5724e2dc4642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:20.743864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad65002-95a4-4d13-b445-5724e2dc4642 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:20.745112 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad65002-95a4-4d13-b445-5724e2dc4642 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:20.746559 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad65002-95a4-4d13-b445-5724e2dc4642 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:20.788767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1370/2768] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 15:20:20 2022] POST /identity/v3/projects => generated 407 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:20:20.791376 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cb19d-207a-463a-a870-999c162f556a admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:20.807719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cb19d-207a-463a-a870-999c162f556a admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:20.845012 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:20.893476 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b958f-a258-477f-aee9-df21a583b594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:20.894255 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b958f-a258-477f-aee9-df21a583b594 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:20.894549 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b958f-a258-477f-aee9-df21a583b594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:20.895556 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b958f-a258-477f-aee9-df21a583b594 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:20.896341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b958f-a258-477f-aee9-df21a583b594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:20.898748 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b958f-a258-477f-aee9-df21a583b594 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:20.935234 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1399/2769] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 15:20:20 2022] POST /identity/v3/users => generated 304 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:20:20.939619 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad65002-95a4-4d13-b445-5724e2dc4642 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:20.951494 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad65002-95a4-4d13-b445-5724e2dc4642 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:20.956742 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ea8c8-c562-439b-a305-e0ddb06a0795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:20.957451 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ea8c8-c562-439b-a305-e0ddb06a0795 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:20.957890 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ea8c8-c562-439b-a305-e0ddb06a0795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:20.958175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ea8c8-c562-439b-a305-e0ddb06a0795 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:20.958959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ea8c8-c562-439b-a305-e0ddb06a0795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:20.960648 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ea8c8-c562-439b-a305-e0ddb06a0795 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:20.983124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1371/2770] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:20:20.988927 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b958f-a258-477f-aee9-df21a583b594 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:21.002695 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b958f-a258-477f-aee9-df21a583b594 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:21.010701 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d461aa-ca4a-4f56-8a83-1dccd3dea646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:21.011694 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d461aa-ca4a-4f56-8a83-1dccd3dea646 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:21.012173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d461aa-ca4a-4f56-8a83-1dccd3dea646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:21.012954 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d461aa-ca4a-4f56-8a83-1dccd3dea646 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:21.014319 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d461aa-ca4a-4f56-8a83-1dccd3dea646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:21.016661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d461aa-ca4a-4f56-8a83-1dccd3dea646 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:21.032505 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1400/2771] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:20:21.039056 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ea8c8-c562-439b-a305-e0ddb06a0795 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:21.054241 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ea8c8-c562-439b-a305-e0ddb06a0795 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:21.062500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4f945-3886-44c3-8360-676a619c81f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:21.063953 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4f945-3886-44c3-8360-676a619c81f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:21.064247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4f945-3886-44c3-8360-676a619c81f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:21.064530 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4f945-3886-44c3-8360-676a619c81f2 admin admin] PATH_INFO: `/v3/projects/31e592605b26464dab5afcfd0aab4988/users/61c0f9966cac49b5b75426e95d21811b/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:21.091294 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4f945-3886-44c3-8360-676a619c81f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31e592605b26464dab5afcfd0aab4988, user_id=61c0f9966cac49b5b75426e95d21811b, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:21.096992 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4f945-3886-44c3-8360-676a619c81f2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:21.121659 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1372/2772] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:20:21 2022] PUT /identity/v3/projects/31e592605b26464dab5afcfd0aab4988/users/61c0f9966cac49b5b75426e95d21811b/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:21.142770 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bd295-5c22-4306-a02b-5694599e451d None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:21.142770 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bd295-5c22-4306-a02b-5694599e451d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:21.143256 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bd295-5c22-4306-a02b-5694599e451d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:21.186075 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-387bd295-5c22-4306-a02b-5694599e451d None None] MFA Rules not processed for user `61c0f9966cac49b5b75426e95d21811b`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:21.189438 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-387bd295-5c22-4306-a02b-5694599e451d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:21.288861 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1401/2773] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:21 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:20:21.294271 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4f945-3886-44c3-8360-676a619c81f2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:21.371468 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:21.397797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4f945-3886-44c3-8360-676a619c81f2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:21.403105 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a22ac-5522-419f-8b09-bb3fe645a708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:21.403795 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a22ac-5522-419f-8b09-bb3fe645a708 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:21.404082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a22ac-5522-419f-8b09-bb3fe645a708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:21.404360 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a22ac-5522-419f-8b09-bb3fe645a708 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:21.405075 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a22ac-5522-419f-8b09-bb3fe645a708 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:21.407071 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a22ac-5522-419f-8b09-bb3fe645a708 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:21.446864 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1373/2774] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 15:20:21 2022] POST /identity/v3/projects => generated 407 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:20:21.451136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bd295-5c22-4306-a02b-5694599e451d None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:21.462930 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bd295-5c22-4306-a02b-5694599e451d None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:21.533908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e9cae-d545-4cf5-87aa-0845d81c5174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:21.534478 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e9cae-d545-4cf5-87aa-0845d81c5174 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:21.534804 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e9cae-d545-4cf5-87aa-0845d81c5174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:21.535093 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e9cae-d545-4cf5-87aa-0845d81c5174 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:21.535810 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e9cae-d545-4cf5-87aa-0845d81c5174 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:21.537777 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e9cae-d545-4cf5-87aa-0845d81c5174 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:21.561058 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1402/2775] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 15:20:21 2022] POST /identity/v3/users => generated 310 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:20:21.565261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a22ac-5522-419f-8b09-bb3fe645a708 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:21.576383 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a22ac-5522-419f-8b09-bb3fe645a708 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:21.581411 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16deb9c0-623b-4ea1-9a9d-310e2f5b2321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:21.582191 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16deb9c0-623b-4ea1-9a9d-310e2f5b2321 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:21.582490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16deb9c0-623b-4ea1-9a9d-310e2f5b2321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:21.582884 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16deb9c0-623b-4ea1-9a9d-310e2f5b2321 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:21.583708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16deb9c0-623b-4ea1-9a9d-310e2f5b2321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:21.587188 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16deb9c0-623b-4ea1-9a9d-310e2f5b2321 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:21.605129 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1374/2776] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 15:20:21.609862 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e9cae-d545-4cf5-87aa-0845d81c5174 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:21.629657 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e9cae-d545-4cf5-87aa-0845d81c5174 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:21.643877 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf077d2-8865-4279-b48a-37c67401670e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:21.646092 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf077d2-8865-4279-b48a-37c67401670e admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:21.646562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf077d2-8865-4279-b48a-37c67401670e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:21.647180 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf077d2-8865-4279-b48a-37c67401670e admin admin] PATH_INFO: `/v3/projects/11a4c5c219b948b79a1ad95a3205e2bf/users/1ecf59ac3c8f4f528a1047c90a5c7940/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:21.674726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf077d2-8865-4279-b48a-37c67401670e admin admin] RBAC: Authorizing `identity:create_grant(project_id=11a4c5c219b948b79a1ad95a3205e2bf, user_id=1ecf59ac3c8f4f528a1047c90a5c7940, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:21.679286 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf077d2-8865-4279-b48a-37c67401670e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:21.713355 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1403/2777] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:20:21 2022] PUT /identity/v3/projects/11a4c5c219b948b79a1ad95a3205e2bf/users/1ecf59ac3c8f4f528a1047c90a5c7940/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:21.718584 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16deb9c0-623b-4ea1-9a9d-310e2f5b2321 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:21.739223 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16deb9c0-623b-4ea1-9a9d-310e2f5b2321 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:21.823093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ecf302-c467-41aa-bc14-5f02e62080ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:21.824226 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ecf302-c467-41aa-bc14-5f02e62080ec admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:21.824723 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ecf302-c467-41aa-bc14-5f02e62080ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:21.825224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ecf302-c467-41aa-bc14-5f02e62080ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:21.826834 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ecf302-c467-41aa-bc14-5f02e62080ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:21.829615 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ecf302-c467-41aa-bc14-5f02e62080ec admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:21.849019 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1375/2778] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 15:20:21.853397 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf077d2-8865-4279-b48a-37c67401670e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:21.859220 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:21.866015 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf077d2-8865-4279-b48a-37c67401670e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:21.872054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb608c-0638-4891-ac10-8399899f8216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:21.873315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb608c-0638-4891-ac10-8399899f8216 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:21.873685 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb608c-0638-4891-ac10-8399899f8216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:21.874050 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb608c-0638-4891-ac10-8399899f8216 admin admin] PATH_INFO: `/v3/projects/11a4c5c219b948b79a1ad95a3205e2bf/users/1ecf59ac3c8f4f528a1047c90a5c7940/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:21.877028 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb608c-0638-4891-ac10-8399899f8216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11a4c5c219b948b79a1ad95a3205e2bf, user_id=1ecf59ac3c8f4f528a1047c90a5c7940, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:21.880113 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb608c-0638-4891-ac10-8399899f8216 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:21.892716 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1404/2779] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:20:21 2022] PUT /identity/v3/projects/11a4c5c219b948b79a1ad95a3205e2bf/users/1ecf59ac3c8f4f528a1047c90a5c7940/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 15:20:21.904401 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ef301-5b01-4816-a763-dd1a3a4a104e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:21.904706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ef301-5b01-4816-a763-dd1a3a4a104e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:21.904995 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ef301-5b01-4816-a763-dd1a3a4a104e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:21.934426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-086ef301-5b01-4816-a763-dd1a3a4a104e None None] MFA Rules not processed for user `1ecf59ac3c8f4f528a1047c90a5c7940`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:21.935853 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-086ef301-5b01-4816-a763-dd1a3a4a104e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:22.018887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1376/2780] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:21 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:20:22.032758 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb608c-0638-4891-ac10-8399899f8216 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:22.034813 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-2bbb608c-0638-4891-ac10-8399899f8216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:22.096713 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb608c-0638-4891-ac10-8399899f8216 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:22.100539 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7517c32b-b991-4969-a227-273b3d9bbb25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:22.101398 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7517c32b-b991-4969-a227-273b3d9bbb25 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:22.101697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7517c32b-b991-4969-a227-273b3d9bbb25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:22.101977 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7517c32b-b991-4969-a227-273b3d9bbb25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:22.112531 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7517c32b-b991-4969-a227-273b3d9bbb25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:22.114041 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7517c32b-b991-4969-a227-273b3d9bbb25 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:22.125929 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1405/2781] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:20:22 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:20:22.282923 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ef301-5b01-4816-a763-dd1a3a4a104e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:22.293251 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ef301-5b01-4816-a763-dd1a3a4a104e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:22.297272 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025fdc6-3fed-484f-aee1-a9eac0eb6f91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:22.298132 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025fdc6-3fed-484f-aee1-a9eac0eb6f91 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:22.298427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025fdc6-3fed-484f-aee1-a9eac0eb6f91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:22.298744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025fdc6-3fed-484f-aee1-a9eac0eb6f91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:22.339439 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:22.367252 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2025fdc6-3fed-484f-aee1-a9eac0eb6f91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:22.369239 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2025fdc6-3fed-484f-aee1-a9eac0eb6f91 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:22.378737 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1377/2782] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:20:22 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:20:25.218124 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7517c32b-b991-4969-a227-273b3d9bbb25 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:25.220263 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7517c32b-b991-4969-a227-273b3d9bbb25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:25.302759 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7517c32b-b991-4969-a227-273b3d9bbb25 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:25.308073 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cce9b1-7d27-437b-8b66-0166d401e41b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:25.308755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cce9b1-7d27-437b-8b66-0166d401e41b service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:25.309098 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cce9b1-7d27-437b-8b66-0166d401e41b service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:25.309379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cce9b1-7d27-437b-8b66-0166d401e41b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:25.311887 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a5cce9b1-7d27-437b-8b66-0166d401e41b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:25.386277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cce9b1-7d27-437b-8b66-0166d401e41b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:25.388674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cce9b1-7d27-437b-8b66-0166d401e41b service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:25.402257 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1406/2783] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:20:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:20:28.771056 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025fdc6-3fed-484f-aee1-a9eac0eb6f91 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:28.773594 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2025fdc6-3fed-484f-aee1-a9eac0eb6f91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:28.799627 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025fdc6-3fed-484f-aee1-a9eac0eb6f91 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:28.810348 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2514d-4a3c-4515-8fa2-7aecd8bad0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:28.812452 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2514d-4a3c-4515-8fa2-7aecd8bad0a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:28.812971 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2514d-4a3c-4515-8fa2-7aecd8bad0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:28.813742 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2514d-4a3c-4515-8fa2-7aecd8bad0a8 admin admin] PATH_INFO: `/v3/users/7a8df9e916ba4542af81e2a29229c1db` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:28.816198 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c2514d-4a3c-4515-8fa2-7aecd8bad0a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a8df9e916ba4542af81e2a29229c1db)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:28.820480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c2514d-4a3c-4515-8fa2-7aecd8bad0a8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:28.912445 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54c2514d-4a3c-4515-8fa2-7aecd8bad0a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a8df9e916ba4542af81e2a29229c1db'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:28.923591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54c2514d-4a3c-4515-8fa2-7aecd8bad0a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a8df9e916ba4542af81e2a29229c1db'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:28.934955 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54c2514d-4a3c-4515-8fa2-7aecd8bad0a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a8df9e916ba4542af81e2a29229c1db'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:28.949590 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54c2514d-4a3c-4515-8fa2-7aecd8bad0a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a8df9e916ba4542af81e2a29229c1db'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:28.950364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54c2514d-4a3c-4515-8fa2-7aecd8bad0a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a8df9e916ba4542af81e2a29229c1db'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:28.961912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1378/2784] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:20:28 2022] DELETE /identity/v3/users/7a8df9e916ba4542af81e2a29229c1db => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:29.208403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cce9b1-7d27-437b-8b66-0166d401e41b service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:29.237324 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cce9b1-7d27-437b-8b66-0166d401e41b service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:29.314470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:29.336088 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8936414-3e27-41ef-953d-64810cc24135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:29.337009 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8936414-3e27-41ef-953d-64810cc24135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:29.337296 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8936414-3e27-41ef-953d-64810cc24135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:29.337578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8936414-3e27-41ef-953d-64810cc24135 admin admin] PATH_INFO: `/v3/projects/5bd34785f6d847ae8cb5be4e07d29c38` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:29.338864 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8936414-3e27-41ef-953d-64810cc24135 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bd34785f6d847ae8cb5be4e07d29c38)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:29.340823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8936414-3e27-41ef-953d-64810cc24135 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:29.397286 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a8936414-3e27-41ef-953d-64810cc24135 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bd34785f6d847ae8cb5be4e07d29c38', 'request_id': 'req-a8936414-3e27-41ef-953d-64810cc24135'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:29.403908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a8936414-3e27-41ef-953d-64810cc24135 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bd34785f6d847ae8cb5be4e07d29c38', 'request_id': 'req-a8936414-3e27-41ef-953d-64810cc24135'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:29.411420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a8936414-3e27-41ef-953d-64810cc24135 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bd34785f6d847ae8cb5be4e07d29c38', 'request_id': 'req-a8936414-3e27-41ef-953d-64810cc24135'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:29.433098 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a8936414-3e27-41ef-953d-64810cc24135 admin admin] The token cache is being invalidate because project 5bd34785f6d847ae8cb5be4e07d29c38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:20:29.433396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a8936414-3e27-41ef-953d-64810cc24135 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:29.435279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a8936414-3e27-41ef-953d-64810cc24135 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:29.438857 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1407/2785] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:20:29 2022] DELETE /identity/v3/projects/5bd34785f6d847ae8cb5be4e07d29c38 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:29.443655 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fb2d0-9c4b-4557-b474-1ddde2fdc766 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:29.445202 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fb2d0-9c4b-4557-b474-1ddde2fdc766 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:29.445503 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fb2d0-9c4b-4557-b474-1ddde2fdc766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:29.475678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-375fb2d0-9c4b-4557-b474-1ddde2fdc766 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:29.476961 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-375fb2d0-9c4b-4557-b474-1ddde2fdc766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:29.553685 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1379/2786] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:20:29.562931 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332164f-4400-46a7-95db-f8c8803271c5 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:29.563446 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332164f-4400-46a7-95db-f8c8803271c5 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:29.563935 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332164f-4400-46a7-95db-f8c8803271c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:29.588195 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-3332164f-4400-46a7-95db-f8c8803271c5 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:29.589748 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3332164f-4400-46a7-95db-f8c8803271c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:29.606818 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1408/2787] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:20:29.611213 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375fb2d0-9c4b-4557-b474-1ddde2fdc766 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:29.627082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375fb2d0-9c4b-4557-b474-1ddde2fdc766 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:29.637386 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a796c3-4253-4271-aaa0-728e52ee5cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:29.638541 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a796c3-4253-4271-aaa0-728e52ee5cef admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:29.639080 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a796c3-4253-4271-aaa0-728e52ee5cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:29.639579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a796c3-4253-4271-aaa0-728e52ee5cef admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:29.641184 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a796c3-4253-4271-aaa0-728e52ee5cef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 15:20:29.641712 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a796c3-4253-4271-aaa0-728e52ee5cef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:29.644736 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a796c3-4253-4271-aaa0-728e52ee5cef admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:29.670724 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1380/2788] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 15:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 15:20:29.674264 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332164f-4400-46a7-95db-f8c8803271c5 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:29.687948 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332164f-4400-46a7-95db-f8c8803271c5 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:29.693235 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c38fc-c2aa-4126-a056-c6d0cadb6c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:29.694069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c38fc-c2aa-4126-a056-c6d0cadb6c4d admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:29.694544 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c38fc-c2aa-4126-a056-c6d0cadb6c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:29.695006 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c38fc-c2aa-4126-a056-c6d0cadb6c4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:29.696651 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c38fc-c2aa-4126-a056-c6d0cadb6c4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:29.699089 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c38fc-c2aa-4126-a056-c6d0cadb6c4d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:29.730147 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1409/2789] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 15:20:29 2022] POST /identity/v3/projects => generated 405 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:20:29.734521 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a796c3-4253-4271-aaa0-728e52ee5cef admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:29.746094 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a796c3-4253-4271-aaa0-728e52ee5cef admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:29.787102 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:29.827964 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283cc4c3-7b20-4b89-88ca-544f536765ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:29.829080 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283cc4c3-7b20-4b89-88ca-544f536765ff admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:29.829671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283cc4c3-7b20-4b89-88ca-544f536765ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:29.830114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283cc4c3-7b20-4b89-88ca-544f536765ff admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:29.831430 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283cc4c3-7b20-4b89-88ca-544f536765ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:29.835059 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283cc4c3-7b20-4b89-88ca-544f536765ff admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:29.884650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1381/2790] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 15:20:29 2022] POST /identity/v3/users => generated 303 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 15:20:29.887677 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c38fc-c2aa-4126-a056-c6d0cadb6c4d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:29.908566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c38fc-c2aa-4126-a056-c6d0cadb6c4d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:29.920642 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215b3fd-8ac4-4ee5-8520-1c80d8c91aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:29.922114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215b3fd-8ac4-4ee5-8520-1c80d8c91aed admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:29.922686 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215b3fd-8ac4-4ee5-8520-1c80d8c91aed admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:29.923350 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215b3fd-8ac4-4ee5-8520-1c80d8c91aed admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:29.924642 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a215b3fd-8ac4-4ee5-8520-1c80d8c91aed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:29.927568 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a215b3fd-8ac4-4ee5-8520-1c80d8c91aed admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:29.951134 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1410/2791] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:20:29.952939 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283cc4c3-7b20-4b89-88ca-544f536765ff admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:29.971294 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283cc4c3-7b20-4b89-88ca-544f536765ff admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:29.980772 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a16f4-b1fd-4c83-a2d9-d2fa00a964e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:29.981891 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a16f4-b1fd-4c83-a2d9-d2fa00a964e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:29.982550 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a16f4-b1fd-4c83-a2d9-d2fa00a964e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:29.982976 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a16f4-b1fd-4c83-a2d9-d2fa00a964e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:29.984415 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a16f4-b1fd-4c83-a2d9-d2fa00a964e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:29.987509 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a16f4-b1fd-4c83-a2d9-d2fa00a964e0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:30.009672 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1382/2792] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 15:20:30.014327 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215b3fd-8ac4-4ee5-8520-1c80d8c91aed admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:30.034995 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215b3fd-8ac4-4ee5-8520-1c80d8c91aed admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:30.044783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43403531-ed8c-4e05-a1b1-51f2738d4fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:30.046264 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43403531-ed8c-4e05-a1b1-51f2738d4fbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:30.046908 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43403531-ed8c-4e05-a1b1-51f2738d4fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:30.047475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43403531-ed8c-4e05-a1b1-51f2738d4fbe admin admin] PATH_INFO: `/v3/projects/11fe1187aa294a37999a5047a1893f10/users/bc2d7c62e3ec4ff2930f82123d70269c/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:30.080546 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43403531-ed8c-4e05-a1b1-51f2738d4fbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=11fe1187aa294a37999a5047a1893f10, user_id=bc2d7c62e3ec4ff2930f82123d70269c, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:30.085159 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43403531-ed8c-4e05-a1b1-51f2738d4fbe admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:30.117548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1411/2793] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:20:30 2022] PUT /identity/v3/projects/11fe1187aa294a37999a5047a1893f10/users/bc2d7c62e3ec4ff2930f82123d70269c/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:30.135407 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c26cbb-a704-45f7-97c5-1e0bbf8b24d0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:30.135815 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c26cbb-a704-45f7-97c5-1e0bbf8b24d0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:30.136315 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c26cbb-a704-45f7-97c5-1e0bbf8b24d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:30.172968 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-04c26cbb-a704-45f7-97c5-1e0bbf8b24d0 None None] MFA Rules not processed for user `bc2d7c62e3ec4ff2930f82123d70269c`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:30.174233 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-04c26cbb-a704-45f7-97c5-1e0bbf8b24d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:30.259814 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1383/2794] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:30 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:20:30.265203 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43403531-ed8c-4e05-a1b1-51f2738d4fbe admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:30.360048 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43403531-ed8c-4e05-a1b1-51f2738d4fbe admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:30.367574 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b3337-5255-474f-a565-2d2d8caadc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:30.368758 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b3337-5255-474f-a565-2d2d8caadc3a admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:30.369240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b3337-5255-474f-a565-2d2d8caadc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:30.369718 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b3337-5255-474f-a565-2d2d8caadc3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:30.371049 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9b3337-5255-474f-a565-2d2d8caadc3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:30.376039 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9b3337-5255-474f-a565-2d2d8caadc3a admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:30.392554 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:30.412038 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1412/2795] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 15:20:30 2022] POST /identity/v3/projects => generated 405 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:20:30.417103 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c26cbb-a704-45f7-97c5-1e0bbf8b24d0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:30.428713 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c26cbb-a704-45f7-97c5-1e0bbf8b24d0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:30.494886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d78f6-fdee-474d-9e5d-2a08eea69b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:30.495618 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d78f6-fdee-474d-9e5d-2a08eea69b4e admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:30.495904 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d78f6-fdee-474d-9e5d-2a08eea69b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:30.496189 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d78f6-fdee-474d-9e5d-2a08eea69b4e admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:30.496899 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d78f6-fdee-474d-9e5d-2a08eea69b4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:30.498921 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d78f6-fdee-474d-9e5d-2a08eea69b4e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:30.525714 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1384/2796] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 15:20:30 2022] POST /identity/v3/users => generated 309 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:20:30.529988 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b3337-5255-474f-a565-2d2d8caadc3a admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:30.541556 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b3337-5255-474f-a565-2d2d8caadc3a admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:30.546608 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f1ca6-a243-4417-9f23-05a5a99b3629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:30.547341 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f1ca6-a243-4417-9f23-05a5a99b3629 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:30.547625 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f1ca6-a243-4417-9f23-05a5a99b3629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:30.547902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f1ca6-a243-4417-9f23-05a5a99b3629 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:30.548657 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f1ca6-a243-4417-9f23-05a5a99b3629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:30.550323 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f1ca6-a243-4417-9f23-05a5a99b3629 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:30.562408 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1413/2797] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:20:30.569038 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d78f6-fdee-474d-9e5d-2a08eea69b4e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:30.587219 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d78f6-fdee-474d-9e5d-2a08eea69b4e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:30.595031 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c9aef-9252-472b-89a2-1080396b8d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:30.596512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c9aef-9252-472b-89a2-1080396b8d94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:30.597024 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c9aef-9252-472b-89a2-1080396b8d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:30.597526 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c9aef-9252-472b-89a2-1080396b8d94 admin admin] PATH_INFO: `/v3/projects/8cc6f9e1d10c45c89837df99742538c3/users/887f778ca63748919bf19e8837853799/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:30.628702 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c9aef-9252-472b-89a2-1080396b8d94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cc6f9e1d10c45c89837df99742538c3, user_id=887f778ca63748919bf19e8837853799, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:30.634391 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c9aef-9252-472b-89a2-1080396b8d94 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:30.655500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1385/2798] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:20:30 2022] PUT /identity/v3/projects/8cc6f9e1d10c45c89837df99742538c3/users/887f778ca63748919bf19e8837853799/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:30.661291 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f1ca6-a243-4417-9f23-05a5a99b3629 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:30.673715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f1ca6-a243-4417-9f23-05a5a99b3629 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:30.778853 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2388780-8390-41c7-8fa1-7194f6b516ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:30.779571 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2388780-8390-41c7-8fa1-7194f6b516ff admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:30.779957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2388780-8390-41c7-8fa1-7194f6b516ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:30.780342 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2388780-8390-41c7-8fa1-7194f6b516ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:30.781187 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2388780-8390-41c7-8fa1-7194f6b516ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:30.783745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2388780-8390-41c7-8fa1-7194f6b516ff admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:30.804875 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1414/2799] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:20:30.811459 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c9aef-9252-472b-89a2-1080396b8d94 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:30.823578 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c9aef-9252-472b-89a2-1080396b8d94 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:30.828679 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c382eaf-521b-4505-8922-304dd8b527e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:30.829634 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c382eaf-521b-4505-8922-304dd8b527e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:30.829915 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c382eaf-521b-4505-8922-304dd8b527e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:30.830190 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c382eaf-521b-4505-8922-304dd8b527e1 admin admin] PATH_INFO: `/v3/projects/8cc6f9e1d10c45c89837df99742538c3/users/887f778ca63748919bf19e8837853799/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:30.832366 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c382eaf-521b-4505-8922-304dd8b527e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cc6f9e1d10c45c89837df99742538c3, user_id=887f778ca63748919bf19e8837853799, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:30.835067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c382eaf-521b-4505-8922-304dd8b527e1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:30.838768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:30.848848 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1386/2800] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:20:30 2022] PUT /identity/v3/projects/8cc6f9e1d10c45c89837df99742538c3/users/887f778ca63748919bf19e8837853799/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:30.861477 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ae792-8467-4d55-a154-3e4d0f70b6d1 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:30.861873 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ae792-8467-4d55-a154-3e4d0f70b6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:30.862198 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ae792-8467-4d55-a154-3e4d0f70b6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:30.893253 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-d81ae792-8467-4d55-a154-3e4d0f70b6d1 None None] MFA Rules not processed for user `887f778ca63748919bf19e8837853799`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:30.894499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-d81ae792-8467-4d55-a154-3e4d0f70b6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:30.984275 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1415/2801] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:30 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 15:20:30.996630 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c382eaf-521b-4505-8922-304dd8b527e1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:30.998591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2c382eaf-521b-4505-8922-304dd8b527e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:31.066086 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c382eaf-521b-4505-8922-304dd8b527e1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:31.070078 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085bb8f2-f940-4a8c-a2bb-f4707dd80e85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:31.071381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085bb8f2-f940-4a8c-a2bb-f4707dd80e85 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:31.071634 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085bb8f2-f940-4a8c-a2bb-f4707dd80e85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:31.071945 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085bb8f2-f940-4a8c-a2bb-f4707dd80e85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:31.091191 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085bb8f2-f940-4a8c-a2bb-f4707dd80e85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:31.092844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085bb8f2-f940-4a8c-a2bb-f4707dd80e85 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:31.104942 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1387/2802] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:20:30 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:20:31.325895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ae792-8467-4d55-a154-3e4d0f70b6d1 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:31.433940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ae792-8467-4d55-a154-3e4d0f70b6d1 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:31.440219 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8930c-cfee-42c8-84d6-e7fd9d9b694b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:31.441337 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8930c-cfee-42c8-84d6-e7fd9d9b694b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:31.441865 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8930c-cfee-42c8-84d6-e7fd9d9b694b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:31.442261 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8930c-cfee-42c8-84d6-e7fd9d9b694b admin admin] PATH_INFO: `/v3/users/bc2d7c62e3ec4ff2930f82123d70269c` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:31.444171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b8930c-cfee-42c8-84d6-e7fd9d9b694b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc2d7c62e3ec4ff2930f82123d70269c)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:31.447427 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b8930c-cfee-42c8-84d6-e7fd9d9b694b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:31.453537 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:31.517972 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-66b8930c-cfee-42c8-84d6-e7fd9d9b694b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc2d7c62e3ec4ff2930f82123d70269c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:31.528895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-66b8930c-cfee-42c8-84d6-e7fd9d9b694b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc2d7c62e3ec4ff2930f82123d70269c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:31.529679 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-66b8930c-cfee-42c8-84d6-e7fd9d9b694b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc2d7c62e3ec4ff2930f82123d70269c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:31.546855 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-66b8930c-cfee-42c8-84d6-e7fd9d9b694b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc2d7c62e3ec4ff2930f82123d70269c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:31.547850 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-66b8930c-cfee-42c8-84d6-e7fd9d9b694b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc2d7c62e3ec4ff2930f82123d70269c'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:31.563759 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1416/2803] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:20:31 2022] DELETE /identity/v3/users/bc2d7c62e3ec4ff2930f82123d70269c => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:31.568769 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb2e80-d27e-4a98-910d-d8bf7530fd10 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:31.569109 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb2e80-d27e-4a98-910d-d8bf7530fd10 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:31.569430 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb2e80-d27e-4a98-910d-d8bf7530fd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:31.594179 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-22bb2e80-d27e-4a98-910d-d8bf7530fd10 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:31.596290 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-22bb2e80-d27e-4a98-910d-d8bf7530fd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:31.702372 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1388/2804] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:20:31.714107 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8930c-cfee-42c8-84d6-e7fd9d9b694b admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:31.715789 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-66b8930c-cfee-42c8-84d6-e7fd9d9b694b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:31.786965 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8930c-cfee-42c8-84d6-e7fd9d9b694b admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:31.792369 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f59e1-e6a8-401b-beb2-13cdc84f6017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:31.793315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f59e1-e6a8-401b-beb2-13cdc84f6017 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:31.793696 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f59e1-e6a8-401b-beb2-13cdc84f6017 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:31.794072 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f59e1-e6a8-401b-beb2-13cdc84f6017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:31.809126 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f59e1-e6a8-401b-beb2-13cdc84f6017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:31.811661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f59e1-e6a8-401b-beb2-13cdc84f6017 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:31.823510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1417/2805] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:20:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:20:32.167806 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb2e80-d27e-4a98-910d-d8bf7530fd10 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:32.169364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-22bb2e80-d27e-4a98-910d-d8bf7530fd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:32.183201 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb2e80-d27e-4a98-910d-d8bf7530fd10 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:32.189275 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86fc08e-3a77-4296-b588-59967331129c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:32.190165 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fc08e-3a77-4296-b588-59967331129c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:32.190460 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fc08e-3a77-4296-b588-59967331129c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:32.190777 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fc08e-3a77-4296-b588-59967331129c admin admin] PATH_INFO: `/v3/projects/11fe1187aa294a37999a5047a1893f10` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:32.192056 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86fc08e-3a77-4296-b588-59967331129c admin admin] RBAC: Authorizing `identity:delete_project(project_id=11fe1187aa294a37999a5047a1893f10)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:32.194476 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86fc08e-3a77-4296-b588-59967331129c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:32.259249 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e86fc08e-3a77-4296-b588-59967331129c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11fe1187aa294a37999a5047a1893f10', 'request_id': 'req-e86fc08e-3a77-4296-b588-59967331129c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:32.266156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e86fc08e-3a77-4296-b588-59967331129c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11fe1187aa294a37999a5047a1893f10', 'request_id': 'req-e86fc08e-3a77-4296-b588-59967331129c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:32.274579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e86fc08e-3a77-4296-b588-59967331129c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11fe1187aa294a37999a5047a1893f10', 'request_id': 'req-e86fc08e-3a77-4296-b588-59967331129c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:32.291064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e86fc08e-3a77-4296-b588-59967331129c admin admin] The token cache is being invalidate because project 11fe1187aa294a37999a5047a1893f10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:20:32.291364 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e86fc08e-3a77-4296-b588-59967331129c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:32.292081 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e86fc08e-3a77-4296-b588-59967331129c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:32.293705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1389/2806] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:20:32 2022] DELETE /identity/v3/projects/11fe1187aa294a37999a5047a1893f10 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:32.298096 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f59e1-e6a8-401b-beb2-13cdc84f6017 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:32.308844 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f59e1-e6a8-401b-beb2-13cdc84f6017 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:32.385456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2142279-d03a-454a-a2dc-1d759b3c6069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:32.386815 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2142279-d03a-454a-a2dc-1d759b3c6069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:32.387210 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2142279-d03a-454a-a2dc-1d759b3c6069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:32.387580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2142279-d03a-454a-a2dc-1d759b3c6069 admin admin] PATH_INFO: `/v3/users/887f778ca63748919bf19e8837853799` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:32.389119 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2142279-d03a-454a-a2dc-1d759b3c6069 admin admin] RBAC: Authorizing `identity:delete_user(user_id=887f778ca63748919bf19e8837853799)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:32.391381 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2142279-d03a-454a-a2dc-1d759b3c6069 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:32.439578 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:32.477937 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e2142279-d03a-454a-a2dc-1d759b3c6069 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '887f778ca63748919bf19e8837853799'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:32.493167 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e2142279-d03a-454a-a2dc-1d759b3c6069 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '887f778ca63748919bf19e8837853799'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:32.493167 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e2142279-d03a-454a-a2dc-1d759b3c6069 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '887f778ca63748919bf19e8837853799'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:32.519155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e2142279-d03a-454a-a2dc-1d759b3c6069 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '887f778ca63748919bf19e8837853799'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:32.520320 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-e2142279-d03a-454a-a2dc-1d759b3c6069 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '887f778ca63748919bf19e8837853799'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:32.550548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1418/2807] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:20:32 2022] DELETE /identity/v3/users/887f778ca63748919bf19e8837853799 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:32.665899 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86fc08e-3a77-4296-b588-59967331129c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:32.668571 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e86fc08e-3a77-4296-b588-59967331129c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:32.690958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86fc08e-3a77-4296-b588-59967331129c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:32.714894 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:32.769192 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43dac8e-ca75-4a9b-84f6-8d8563d29541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:32.769956 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43dac8e-ca75-4a9b-84f6-8d8563d29541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:32.770250 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43dac8e-ca75-4a9b-84f6-8d8563d29541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:32.770531 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43dac8e-ca75-4a9b-84f6-8d8563d29541 admin admin] PATH_INFO: `/v3/users/61c0f9966cac49b5b75426e95d21811b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:32.772155 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43dac8e-ca75-4a9b-84f6-8d8563d29541 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61c0f9966cac49b5b75426e95d21811b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:32.774611 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43dac8e-ca75-4a9b-84f6-8d8563d29541 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:32.838899 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c43dac8e-ca75-4a9b-84f6-8d8563d29541 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61c0f9966cac49b5b75426e95d21811b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:32.848164 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c43dac8e-ca75-4a9b-84f6-8d8563d29541 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61c0f9966cac49b5b75426e95d21811b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:32.858439 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c43dac8e-ca75-4a9b-84f6-8d8563d29541 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61c0f9966cac49b5b75426e95d21811b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:32.871740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c43dac8e-ca75-4a9b-84f6-8d8563d29541 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61c0f9966cac49b5b75426e95d21811b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:32.872687 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c43dac8e-ca75-4a9b-84f6-8d8563d29541 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61c0f9966cac49b5b75426e95d21811b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:32.879751 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1390/2808] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:20:32 2022] DELETE /identity/v3/users/61c0f9966cac49b5b75426e95d21811b => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:32.884445 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691d54e-3ef1-4d0c-9d13-da118fea4f1c None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:32.884738 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691d54e-3ef1-4d0c-9d13-da118fea4f1c None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:32.885020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691d54e-3ef1-4d0c-9d13-da118fea4f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:32.905227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a691d54e-3ef1-4d0c-9d13-da118fea4f1c None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:32.906586 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a691d54e-3ef1-4d0c-9d13-da118fea4f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:32.937512 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43dac8e-ca75-4a9b-84f6-8d8563d29541 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:32.939123 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c43dac8e-ca75-4a9b-84f6-8d8563d29541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:32.999305 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1419/2809] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:20:33.009605 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691d54e-3ef1-4d0c-9d13-da118fea4f1c None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:33.011277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a691d54e-3ef1-4d0c-9d13-da118fea4f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:33.031073 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43dac8e-ca75-4a9b-84f6-8d8563d29541 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:33.036738 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1464aa7-687c-4545-ab0e-c039226c42f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:33.037768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1464aa7-687c-4545-ab0e-c039226c42f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:33.038059 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1464aa7-687c-4545-ab0e-c039226c42f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:33.038338 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1464aa7-687c-4545-ab0e-c039226c42f2 admin admin] PATH_INFO: `/v3/projects/8cc6f9e1d10c45c89837df99742538c3` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:33.040404 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1464aa7-687c-4545-ab0e-c039226c42f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cc6f9e1d10c45c89837df99742538c3)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:33.043109 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1464aa7-687c-4545-ab0e-c039226c42f2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:33.107182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:33.115521 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a1464aa7-687c-4545-ab0e-c039226c42f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cc6f9e1d10c45c89837df99742538c3', 'request_id': 'req-a1464aa7-687c-4545-ab0e-c039226c42f2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:33.122706 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691d54e-3ef1-4d0c-9d13-da118fea4f1c None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:33.128120 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a1464aa7-687c-4545-ab0e-c039226c42f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cc6f9e1d10c45c89837df99742538c3', 'request_id': 'req-a1464aa7-687c-4545-ab0e-c039226c42f2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:33.138253 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a1464aa7-687c-4545-ab0e-c039226c42f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cc6f9e1d10c45c89837df99742538c3', 'request_id': 'req-a1464aa7-687c-4545-ab0e-c039226c42f2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:33.153011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a1464aa7-687c-4545-ab0e-c039226c42f2 admin admin] The token cache is being invalidate because project 8cc6f9e1d10c45c89837df99742538c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:20:33.153304 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a1464aa7-687c-4545-ab0e-c039226c42f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:33.154007 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a1464aa7-687c-4545-ab0e-c039226c42f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:33.155751 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1391/2810] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:20:32 2022] DELETE /identity/v3/projects/8cc6f9e1d10c45c89837df99742538c3 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 15:20:33.162566 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e805dfd-a46e-4eb2-9e83-ee4eab602d04 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:33.162920 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e805dfd-a46e-4eb2-9e83-ee4eab602d04 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:33.163269 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e805dfd-a46e-4eb2-9e83-ee4eab602d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:33.184516 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9e805dfd-a46e-4eb2-9e83-ee4eab602d04 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:33.185773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9e805dfd-a46e-4eb2-9e83-ee4eab602d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:33.189936 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca5761-94b4-454e-b5f6-02e666104848 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:33.190679 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca5761-94b4-454e-b5f6-02e666104848 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:33.191163 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca5761-94b4-454e-b5f6-02e666104848 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:33.191492 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca5761-94b4-454e-b5f6-02e666104848 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:33.194029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-52ca5761-94b4-454e-b5f6-02e666104848 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:33.210232 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:33.254418 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1392/2811] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:20:33.264600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e4256-6b6d-467b-9c63-b9df91eb0ff6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:33.264885 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e4256-6b6d-467b-9c63-b9df91eb0ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:33.265161 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e4256-6b6d-467b-9c63-b9df91eb0ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:33.268783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ca5761-94b4-454e-b5f6-02e666104848 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:33.270250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ca5761-94b4-454e-b5f6-02e666104848 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:33.281479 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1420/2812] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:20:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 15:20:33.285824 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-431e4256-6b6d-467b-9c63-b9df91eb0ff6 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:33.287105 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-431e4256-6b6d-467b-9c63-b9df91eb0ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:33.296551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1393/2813] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 15:20:33.301260 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca5761-94b4-454e-b5f6-02e666104848 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:33.311903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca5761-94b4-454e-b5f6-02e666104848 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:33.318449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61276668-f915-4c4e-bf51-3662c701202a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:33.318841 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61276668-f915-4c4e-bf51-3662c701202a admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:33.319132 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61276668-f915-4c4e-bf51-3662c701202a admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:33.319416 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61276668-f915-4c4e-bf51-3662c701202a admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:33.320199 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61276668-f915-4c4e-bf51-3662c701202a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 15:20:33.320514 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61276668-f915-4c4e-bf51-3662c701202a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:33.322781 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61276668-f915-4c4e-bf51-3662c701202a admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:33.335433 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1421/2814] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 15:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 15:20:33.340535 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e4256-6b6d-467b-9c63-b9df91eb0ff6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:33.350836 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e4256-6b6d-467b-9c63-b9df91eb0ff6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:33.356246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77c29a-c2fd-40de-8ff9-e1304d8f29ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:33.356989 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77c29a-c2fd-40de-8ff9-e1304d8f29ea admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:33.357318 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77c29a-c2fd-40de-8ff9-e1304d8f29ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:33.357706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77c29a-c2fd-40de-8ff9-e1304d8f29ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:33.358436 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77c29a-c2fd-40de-8ff9-e1304d8f29ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:33.360533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77c29a-c2fd-40de-8ff9-e1304d8f29ea admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:33.407401 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1394/2815] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 15:20:33 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:20:33.409222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61276668-f915-4c4e-bf51-3662c701202a admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:33.426186 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61276668-f915-4c4e-bf51-3662c701202a admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:33.491664 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb292b30-11cf-44a6-918d-0d9ae2320e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:33.493268 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb292b30-11cf-44a6-918d-0d9ae2320e6e admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:33.493561 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb292b30-11cf-44a6-918d-0d9ae2320e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:33.493961 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb292b30-11cf-44a6-918d-0d9ae2320e6e admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:33.494747 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb292b30-11cf-44a6-918d-0d9ae2320e6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:33.496755 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb292b30-11cf-44a6-918d-0d9ae2320e6e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:33.518472 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1422/2816] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 15:20:33 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:20:33.522894 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77c29a-c2fd-40de-8ff9-e1304d8f29ea admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:33.535718 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77c29a-c2fd-40de-8ff9-e1304d8f29ea admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:33.540754 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6429f-a91d-4f15-a101-7e6f13c1988a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:33.541731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6429f-a91d-4f15-a101-7e6f13c1988a admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:33.542088 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6429f-a91d-4f15-a101-7e6f13c1988a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:33.542593 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6429f-a91d-4f15-a101-7e6f13c1988a admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:33.543585 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6429f-a91d-4f15-a101-7e6f13c1988a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:33.545615 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6429f-a91d-4f15-a101-7e6f13c1988a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:33.558720 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1395/2817] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:20:33.563052 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb292b30-11cf-44a6-918d-0d9ae2320e6e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:33.574402 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb292b30-11cf-44a6-918d-0d9ae2320e6e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:33.579905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90371f7e-0f4f-4231-b5d8-3d4cf5e860ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:33.580684 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90371f7e-0f4f-4231-b5d8-3d4cf5e860ca admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:33.580993 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90371f7e-0f4f-4231-b5d8-3d4cf5e860ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:33.581270 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90371f7e-0f4f-4231-b5d8-3d4cf5e860ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:33.582012 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90371f7e-0f4f-4231-b5d8-3d4cf5e860ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:33.583715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90371f7e-0f4f-4231-b5d8-3d4cf5e860ca admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:33.597029 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6429f-a91d-4f15-a101-7e6f13c1988a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:33.599264 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0bb6429f-a91d-4f15-a101-7e6f13c1988a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:33.599403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1423/2818] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:20:33.603387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90371f7e-0f4f-4231-b5d8-3d4cf5e860ca admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:33.620887 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90371f7e-0f4f-4231-b5d8-3d4cf5e860ca admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:33.627413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bceaff3-885b-46f7-ac88-47637c410416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:33.627599 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6429f-a91d-4f15-a101-7e6f13c1988a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:33.628246 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bceaff3-885b-46f7-ac88-47637c410416 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:33.628524 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bceaff3-885b-46f7-ac88-47637c410416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:33.628804 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bceaff3-885b-46f7-ac88-47637c410416 admin admin] PATH_INFO: `/v3/projects/e03724fb968b40808429b35a689bea3e/users/7e92cefa239743299a5bb1687dbc2994/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:33.634009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe88f1f-b3c3-4697-9952-b66f91a34ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:33.634477 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe88f1f-b3c3-4697-9952-b66f91a34ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:33.634925 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe88f1f-b3c3-4697-9952-b66f91a34ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:33.635239 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe88f1f-b3c3-4697-9952-b66f91a34ee1 admin admin] PATH_INFO: `/v3/projects/31e592605b26464dab5afcfd0aab4988` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:33.636678 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe88f1f-b3c3-4697-9952-b66f91a34ee1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31e592605b26464dab5afcfd0aab4988)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:33.638827 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe88f1f-b3c3-4697-9952-b66f91a34ee1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:33.648903 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bceaff3-885b-46f7-ac88-47637c410416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e03724fb968b40808429b35a689bea3e, user_id=7e92cefa239743299a5bb1687dbc2994, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:33.651763 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bceaff3-885b-46f7-ac88-47637c410416 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:33.669773 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1424/2819] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:20:33 2022] PUT /identity/v3/projects/e03724fb968b40808429b35a689bea3e/users/7e92cefa239743299a5bb1687dbc2994/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 15:20:33.681935 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a594df-a4a2-4cd0-bf32-acef982ed35a None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:33.682230 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a594df-a4a2-4cd0-bf32-acef982ed35a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:33.682720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a594df-a4a2-4cd0-bf32-acef982ed35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:33.723481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-a9a594df-a4a2-4cd0-bf32-acef982ed35a None None] MFA Rules not processed for user `7e92cefa239743299a5bb1687dbc2994`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:33.724858 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a9a594df-a4a2-4cd0-bf32-acef982ed35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:33.726416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dbe88f1f-b3c3-4697-9952-b66f91a34ee1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31e592605b26464dab5afcfd0aab4988', 'request_id': 'req-dbe88f1f-b3c3-4697-9952-b66f91a34ee1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:33.737319 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dbe88f1f-b3c3-4697-9952-b66f91a34ee1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31e592605b26464dab5afcfd0aab4988', 'request_id': 'req-dbe88f1f-b3c3-4697-9952-b66f91a34ee1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:33.751633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dbe88f1f-b3c3-4697-9952-b66f91a34ee1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31e592605b26464dab5afcfd0aab4988', 'request_id': 'req-dbe88f1f-b3c3-4697-9952-b66f91a34ee1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:33.776858 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dbe88f1f-b3c3-4697-9952-b66f91a34ee1 admin admin] The token cache is being invalidate because project 31e592605b26464dab5afcfd0aab4988 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:20:33.777155 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dbe88f1f-b3c3-4697-9952-b66f91a34ee1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:33.777893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dbe88f1f-b3c3-4697-9952-b66f91a34ee1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:33.779942 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1396/2820] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:20:33 2022] DELETE /identity/v3/projects/31e592605b26464dab5afcfd0aab4988 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:33.784733 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe88f1f-b3c3-4697-9952-b66f91a34ee1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:33.787938 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dbe88f1f-b3c3-4697-9952-b66f91a34ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:33.797912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:33.832457 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:33.838123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1425/2821] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 15:20:33.840902 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a594df-a4a2-4cd0-bf32-acef982ed35a None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:33.842478 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a9a594df-a4a2-4cd0-bf32-acef982ed35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:33.926865 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe88f1f-b3c3-4697-9952-b66f91a34ee1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:33.935846 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a88c1-16a0-4c67-8dbd-d4ea48a35633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:33.937168 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a88c1-16a0-4c67-8dbd-d4ea48a35633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:33.938271 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a88c1-16a0-4c67-8dbd-d4ea48a35633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:33.939110 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a88c1-16a0-4c67-8dbd-d4ea48a35633 admin admin] PATH_INFO: `/v3/users/1ecf59ac3c8f4f528a1047c90a5c7940` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:33.940920 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43a88c1-16a0-4c67-8dbd-d4ea48a35633 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ecf59ac3c8f4f528a1047c90a5c7940)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:33.943417 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43a88c1-16a0-4c67-8dbd-d4ea48a35633 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:33.943654 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a594df-a4a2-4cd0-bf32-acef982ed35a None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:33.948740 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44588ee-0192-4f96-930e-ff71109df209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:33.949420 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44588ee-0192-4f96-930e-ff71109df209 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:33.949705 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44588ee-0192-4f96-930e-ff71109df209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:33.949979 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44588ee-0192-4f96-930e-ff71109df209 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:33.950730 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44588ee-0192-4f96-930e-ff71109df209 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:33.952746 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44588ee-0192-4f96-930e-ff71109df209 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:33.990439 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1426/2822] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 15:20:33 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 15:20:33.995875 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44588ee-0192-4f96-930e-ff71109df209 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:34.007314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f43a88c1-16a0-4c67-8dbd-d4ea48a35633 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ecf59ac3c8f4f528a1047c90a5c7940'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:34.016101 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f43a88c1-16a0-4c67-8dbd-d4ea48a35633 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ecf59ac3c8f4f528a1047c90a5c7940'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:34.024150 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f43a88c1-16a0-4c67-8dbd-d4ea48a35633 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ecf59ac3c8f4f528a1047c90a5c7940'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:34.037310 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f43a88c1-16a0-4c67-8dbd-d4ea48a35633 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ecf59ac3c8f4f528a1047c90a5c7940'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:34.038160 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f43a88c1-16a0-4c67-8dbd-d4ea48a35633 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ecf59ac3c8f4f528a1047c90a5c7940'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:34.046309 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1397/2823] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:20:33 2022] DELETE /identity/v3/users/1ecf59ac3c8f4f528a1047c90a5c7940 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 15:20:34.101242 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44588ee-0192-4f96-930e-ff71109df209 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:34.197121 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9346c-9627-4682-9f15-e7a5b5befa6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:34.197740 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9346c-9627-4682-9f15-e7a5b5befa6e admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:34.198106 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9346c-9627-4682-9f15-e7a5b5befa6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:34.198314 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9346c-9627-4682-9f15-e7a5b5befa6e admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:34.199143 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d9346c-9627-4682-9f15-e7a5b5befa6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:34.201500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d9346c-9627-4682-9f15-e7a5b5befa6e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:34.226088 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1427/2824] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 15:20:33 2022] POST /identity/v3/users => generated 295 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 15:20:34.230700 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a88c1-16a0-4c67-8dbd-d4ea48a35633 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:34.232295 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f43a88c1-16a0-4c67-8dbd-d4ea48a35633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:34.249035 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a88c1-16a0-4c67-8dbd-d4ea48a35633 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:34.259693 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ab0aa-28b5-429b-8b0e-5c39f54e482e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:34.260912 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ab0aa-28b5-429b-8b0e-5c39f54e482e admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:34.262023 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ab0aa-28b5-429b-8b0e-5c39f54e482e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:34.263081 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ab0aa-28b5-429b-8b0e-5c39f54e482e admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:34.264602 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5ab0aa-28b5-429b-8b0e-5c39f54e482e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:34.267768 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5ab0aa-28b5-429b-8b0e-5c39f54e482e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:34.294201 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1398/2825] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 15:20:34.296911 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9346c-9627-4682-9f15-e7a5b5befa6e admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:34.307242 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:34.314922 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9346c-9627-4682-9f15-e7a5b5befa6e admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:34.324739 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc056999-9479-4978-8190-bc03691efd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:34.326418 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc056999-9479-4978-8190-bc03691efd56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:34.327116 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc056999-9479-4978-8190-bc03691efd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:34.327601 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc056999-9479-4978-8190-bc03691efd56 admin admin] PATH_INFO: `/v3/projects/44736e52b3984b838a71040d4f5f8352/users/bcab8a621e0c498a89e1c8f006789d39/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:34.355707 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc056999-9479-4978-8190-bc03691efd56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44736e52b3984b838a71040d4f5f8352, user_id=bcab8a621e0c498a89e1c8f006789d39, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:34.361312 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc056999-9479-4978-8190-bc03691efd56 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:34.386801 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1428/2826] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:20:34 2022] PUT /identity/v3/projects/44736e52b3984b838a71040d4f5f8352/users/bcab8a621e0c498a89e1c8f006789d39/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:34.392262 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ab0aa-28b5-429b-8b0e-5c39f54e482e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:34.408866 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc056999-9479-4978-8190-bc03691efd56 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:34.410597 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ab0aa-28b5-429b-8b0e-5c39f54e482e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:34.412660 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-dc056999-9479-4978-8190-bc03691efd56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:34.504652 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ca09b-c6b8-4976-90b0-06dbf15fbb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:34.505551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ca09b-c6b8-4976-90b0-06dbf15fbb24 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:34.505980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ca09b-c6b8-4976-90b0-06dbf15fbb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:34.506331 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ca09b-c6b8-4976-90b0-06dbf15fbb24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:34.507224 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266ca09b-c6b8-4976-90b0-06dbf15fbb24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:34.509013 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266ca09b-c6b8-4976-90b0-06dbf15fbb24 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:34.524417 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc056999-9479-4978-8190-bc03691efd56 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:34.528515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1399/2827] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 15:20:34.531332 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffe3d6-1b9f-46e5-a2ae-cae9d346a0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:34.531547 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ca09b-c6b8-4976-90b0-06dbf15fbb24 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:34.531702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffe3d6-1b9f-46e5-a2ae-cae9d346a0c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:34.531702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffe3d6-1b9f-46e5-a2ae-cae9d346a0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:34.531702 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffe3d6-1b9f-46e5-a2ae-cae9d346a0c8 admin admin] PATH_INFO: `/v3/projects/11a4c5c219b948b79a1ad95a3205e2bf` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:34.533060 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffe3d6-1b9f-46e5-a2ae-cae9d346a0c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11a4c5c219b948b79a1ad95a3205e2bf)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:34.537623 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffe3d6-1b9f-46e5-a2ae-cae9d346a0c8 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:34.544803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ca09b-c6b8-4976-90b0-06dbf15fbb24 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:34.550821 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc068b4c-32df-417f-82fa-a5240f07917a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:34.551835 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc068b4c-32df-417f-82fa-a5240f07917a admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:34.552132 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc068b4c-32df-417f-82fa-a5240f07917a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:34.552410 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc068b4c-32df-417f-82fa-a5240f07917a admin admin] PATH_INFO: `/v3/projects/44736e52b3984b838a71040d4f5f8352/users/bcab8a621e0c498a89e1c8f006789d39/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:34.555079 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc068b4c-32df-417f-82fa-a5240f07917a admin admin] RBAC: Authorizing `identity:create_grant(project_id=44736e52b3984b838a71040d4f5f8352, user_id=bcab8a621e0c498a89e1c8f006789d39, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:34.557778 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc068b4c-32df-417f-82fa-a5240f07917a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:34.570531 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1400/2828] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:20:34 2022] PUT /identity/v3/projects/44736e52b3984b838a71040d4f5f8352/users/bcab8a621e0c498a89e1c8f006789d39/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:34.590811 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee937e0-f50c-4e43-a4c2-78262b9d7c11 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:34.591301 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee937e0-f50c-4e43-a4c2-78262b9d7c11 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:34.591754 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee937e0-f50c-4e43-a4c2-78262b9d7c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:34.599152 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f1ffe3d6-1b9f-46e5-a2ae-cae9d346a0c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11a4c5c219b948b79a1ad95a3205e2bf', 'request_id': 'req-f1ffe3d6-1b9f-46e5-a2ae-cae9d346a0c8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:34.606158 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:34.607066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f1ffe3d6-1b9f-46e5-a2ae-cae9d346a0c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11a4c5c219b948b79a1ad95a3205e2bf', 'request_id': 'req-f1ffe3d6-1b9f-46e5-a2ae-cae9d346a0c8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:34.616935 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f1ffe3d6-1b9f-46e5-a2ae-cae9d346a0c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11a4c5c219b948b79a1ad95a3205e2bf', 'request_id': 'req-f1ffe3d6-1b9f-46e5-a2ae-cae9d346a0c8'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:34.632370 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f1ffe3d6-1b9f-46e5-a2ae-cae9d346a0c8 admin admin] The token cache is being invalidate because project 11a4c5c219b948b79a1ad95a3205e2bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:20:34.632652 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f1ffe3d6-1b9f-46e5-a2ae-cae9d346a0c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:34.633826 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f1ffe3d6-1b9f-46e5-a2ae-cae9d346a0c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:34.635491 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1429/2829] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:20:34 2022] DELETE /identity/v3/projects/11a4c5c219b948b79a1ad95a3205e2bf => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 15:20:34.636073 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-fee937e0-f50c-4e43-a4c2-78262b9d7c11 None None] MFA Rules not processed for user `bcab8a621e0c498a89e1c8f006789d39`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:34.638229 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fee937e0-f50c-4e43-a4c2-78262b9d7c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:34.642326 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baeb269-c9e2-4208-9e42-d733dd45ecac None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:34.642611 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baeb269-c9e2-4208-9e42-d733dd45ecac None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:34.642911 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baeb269-c9e2-4208-9e42-d733dd45ecac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:34.671795 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-1baeb269-c9e2-4208-9e42-d733dd45ecac None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:34.673221 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1baeb269-c9e2-4208-9e42-d733dd45ecac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:34.735339 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1401/2830] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:34 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:20:34.748177 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee937e0-f50c-4e43-a4c2-78262b9d7c11 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:34.749692 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1430/2831] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 15:20:34.749998 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fee937e0-f50c-4e43-a4c2-78262b9d7c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:34.759728 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb83e93-1cc1-4feb-8f12-1bcecbac9051 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:34.759728 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb83e93-1cc1-4feb-8f12-1bcecbac9051 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:34.760940 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb83e93-1cc1-4feb-8f12-1bcecbac9051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:34.780110 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-6cb83e93-1cc1-4feb-8f12-1bcecbac9051 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:34.781519 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6cb83e93-1cc1-4feb-8f12-1bcecbac9051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:34.790956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1431/2832] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 15:20:34.797780 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb83e93-1cc1-4feb-8f12-1bcecbac9051 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:34.803541 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:34.807935 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb83e93-1cc1-4feb-8f12-1bcecbac9051 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:34.815609 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a29353-f153-413b-bdc2-4159b7237fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:34.816570 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a29353-f153-413b-bdc2-4159b7237fb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:34.816837 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a29353-f153-413b-bdc2-4159b7237fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:34.817776 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a29353-f153-413b-bdc2-4159b7237fb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:34.818711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a29353-f153-413b-bdc2-4159b7237fb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 15:20:34.819066 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a29353-f153-413b-bdc2-4159b7237fb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:34.821788 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a29353-f153-413b-bdc2-4159b7237fb6 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:34.822936 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee937e0-f50c-4e43-a4c2-78262b9d7c11 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:34.828549 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a114e-3cd4-473e-8ade-27f6bc6c1e11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:34.829651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a114e-3cd4-473e-8ade-27f6bc6c1e11 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:34.830166 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a114e-3cd4-473e-8ade-27f6bc6c1e11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:34.830717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a114e-3cd4-473e-8ade-27f6bc6c1e11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:34.833982 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1f6a114e-3cd4-473e-8ade-27f6bc6c1e11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:34.837115 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1432/2833] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 15:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 15:20:34.841569 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a29353-f153-413b-bdc2-4159b7237fb6 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:34.851890 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a29353-f153-413b-bdc2-4159b7237fb6 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:34.856843 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d349c-3388-4019-9874-3195190899d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:34.857514 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d349c-3388-4019-9874-3195190899d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:34.857796 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d349c-3388-4019-9874-3195190899d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:34.858761 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d349c-3388-4019-9874-3195190899d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:34.858980 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09d349c-3388-4019-9874-3195190899d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:34.860807 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09d349c-3388-4019-9874-3195190899d3 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:34.898239 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1433/2834] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 15:20:34 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 15:20:34.903244 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d349c-3388-4019-9874-3195190899d3 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:34.923790 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a114e-3cd4-473e-8ade-27f6bc6c1e11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:34.925270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a114e-3cd4-473e-8ade-27f6bc6c1e11 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:34.936717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1402/2835] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:20:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 15:20:34.992626 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d349c-3388-4019-9874-3195190899d3 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:35.000986 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545128d-8290-4a5b-ab7f-51d9c255c677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:35.002167 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545128d-8290-4a5b-ab7f-51d9c255c677 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:35.002705 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545128d-8290-4a5b-ab7f-51d9c255c677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:35.003213 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545128d-8290-4a5b-ab7f-51d9c255c677 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:35.004383 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545128d-8290-4a5b-ab7f-51d9c255c677 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:35.007508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545128d-8290-4a5b-ab7f-51d9c255c677 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:35.041713 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1434/2836] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 15:20:34 2022] POST /identity/v3/users => generated 291 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 15:20:35.047465 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a114e-3cd4-473e-8ade-27f6bc6c1e11 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:35.070386 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a114e-3cd4-473e-8ade-27f6bc6c1e11 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:35.080648 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e13c36-ec42-4785-bc16-42c2d7afdb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:35.081850 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e13c36-ec42-4785-bc16-42c2d7afdb58 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:35.082357 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e13c36-ec42-4785-bc16-42c2d7afdb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:35.082872 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e13c36-ec42-4785-bc16-42c2d7afdb58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:35.084207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e13c36-ec42-4785-bc16-42c2d7afdb58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:35.088199 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e13c36-ec42-4785-bc16-42c2d7afdb58 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:35.107738 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1403/2837] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 15:20:35.111873 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545128d-8290-4a5b-ab7f-51d9c255c677 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:35.137326 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545128d-8290-4a5b-ab7f-51d9c255c677 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:35.145894 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451c908-44ea-4efc-bcf1-e13d2791e99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:35.146744 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451c908-44ea-4efc-bcf1-e13d2791e99c admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:35.146893 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451c908-44ea-4efc-bcf1-e13d2791e99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:35.147218 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451c908-44ea-4efc-bcf1-e13d2791e99c admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:35.147979 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a451c908-44ea-4efc-bcf1-e13d2791e99c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:35.150145 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a451c908-44ea-4efc-bcf1-e13d2791e99c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:35.168700 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1435/2838] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:20:35.169426 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e13c36-ec42-4785-bc16-42c2d7afdb58 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:35.180452 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e13c36-ec42-4785-bc16-42c2d7afdb58 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:35.185345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d18ae-dcde-4054-9a2f-18660418c772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:35.186265 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d18ae-dcde-4054-9a2f-18660418c772 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:35.186551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d18ae-dcde-4054-9a2f-18660418c772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:35.186872 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d18ae-dcde-4054-9a2f-18660418c772 admin admin] PATH_INFO: `/v3/projects/bd7690e7a8be4465b961ff5ae495a580/users/050b371b7ade41b2b58bdc4edf97ef6f/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:35.205471 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d18ae-dcde-4054-9a2f-18660418c772 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd7690e7a8be4465b961ff5ae495a580, user_id=050b371b7ade41b2b58bdc4edf97ef6f, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:35.208479 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d18ae-dcde-4054-9a2f-18660418c772 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:35.228546 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1404/2839] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:20:35 2022] PUT /identity/v3/projects/bd7690e7a8be4465b961ff5ae495a580/users/050b371b7ade41b2b58bdc4edf97ef6f/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:35.240449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0cb05-9022-4268-a6c1-c2f497b917a3 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:35.240798 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0cb05-9022-4268-a6c1-c2f497b917a3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:35.241033 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0cb05-9022-4268-a6c1-c2f497b917a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:35.288205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-4ec0cb05-9022-4268-a6c1-c2f497b917a3 None None] MFA Rules not processed for user `050b371b7ade41b2b58bdc4edf97ef6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:35.291679 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4ec0cb05-9022-4268-a6c1-c2f497b917a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:35.381273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1436/2840] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:20:35.386569 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d18ae-dcde-4054-9a2f-18660418c772 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:35.405886 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:35.465026 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d18ae-dcde-4054-9a2f-18660418c772 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:35.472105 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4163b-adf1-44ce-bc48-8e2e48daf228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:35.472791 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4163b-adf1-44ce-bc48-8e2e48daf228 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:35.473075 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4163b-adf1-44ce-bc48-8e2e48daf228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:35.473352 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4163b-adf1-44ce-bc48-8e2e48daf228 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:35.474057 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4163b-adf1-44ce-bc48-8e2e48daf228 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:35.476018 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4163b-adf1-44ce-bc48-8e2e48daf228 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:35.503502 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1405/2841] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 15:20:35 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:20:35.507630 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0cb05-9022-4268-a6c1-c2f497b917a3 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:35.518499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0cb05-9022-4268-a6c1-c2f497b917a3 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:35.554156 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:35.581059 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4e3d4-0d70-434e-b3a6-60d9d5a6e3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:35.581869 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4e3d4-0d70-434e-b3a6-60d9d5a6e3bb admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:35.582158 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4e3d4-0d70-434e-b3a6-60d9d5a6e3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:35.582446 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4e3d4-0d70-434e-b3a6-60d9d5a6e3bb admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:35.583513 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba4e3d4-0d70-434e-b3a6-60d9d5a6e3bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:35.585956 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba4e3d4-0d70-434e-b3a6-60d9d5a6e3bb admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:35.608785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1437/2842] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 15:20:35 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:20:35.618366 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4163b-adf1-44ce-bc48-8e2e48daf228 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:35.629538 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4163b-adf1-44ce-bc48-8e2e48daf228 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:35.634316 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78ca2f-3de2-4728-8c55-849c3e2fe6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:35.635039 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78ca2f-3de2-4728-8c55-849c3e2fe6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:35.635315 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78ca2f-3de2-4728-8c55-849c3e2fe6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:35.635591 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78ca2f-3de2-4728-8c55-849c3e2fe6b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:35.636327 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78ca2f-3de2-4728-8c55-849c3e2fe6b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:35.637949 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78ca2f-3de2-4728-8c55-849c3e2fe6b9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:35.651157 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1406/2843] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:20:35.655769 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4e3d4-0d70-434e-b3a6-60d9d5a6e3bb admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:35.667426 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4e3d4-0d70-434e-b3a6-60d9d5a6e3bb admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:35.672200 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17bce5-fa01-4998-a5f7-5a848e27b7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:35.673096 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17bce5-fa01-4998-a5f7-5a848e27b7cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:35.673383 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17bce5-fa01-4998-a5f7-5a848e27b7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:35.673661 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17bce5-fa01-4998-a5f7-5a848e27b7cd admin admin] PATH_INFO: `/v3/projects/c30c5e6090d34ef4bc14f047af450c6e/users/70bf791301ef47a4ba0eceeab850a88e/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:35.691602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e17bce5-fa01-4998-a5f7-5a848e27b7cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c30c5e6090d34ef4bc14f047af450c6e, user_id=70bf791301ef47a4ba0eceeab850a88e, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:35.694538 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e17bce5-fa01-4998-a5f7-5a848e27b7cd admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:35.708288 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1438/2844] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:20:35 2022] PUT /identity/v3/projects/c30c5e6090d34ef4bc14f047af450c6e/users/70bf791301ef47a4ba0eceeab850a88e/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:35.713076 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78ca2f-3de2-4728-8c55-849c3e2fe6b9 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:35.723308 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78ca2f-3de2-4728-8c55-849c3e2fe6b9 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:35.786434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4029ef6-bdf4-42d4-8eb9-5bd12b508e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:35.787207 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4029ef6-bdf4-42d4-8eb9-5bd12b508e59 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:35.787490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4029ef6-bdf4-42d4-8eb9-5bd12b508e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:35.787766 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4029ef6-bdf4-42d4-8eb9-5bd12b508e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:35.788498 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4029ef6-bdf4-42d4-8eb9-5bd12b508e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:35.790816 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4029ef6-bdf4-42d4-8eb9-5bd12b508e59 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:35.802399 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1407/2845] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:20:35.807452 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17bce5-fa01-4998-a5f7-5a848e27b7cd admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:35.818783 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17bce5-fa01-4998-a5f7-5a848e27b7cd admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:35.823952 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda64e2b-ecce-4f77-97eb-3f6bea0e5be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:35.824862 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda64e2b-ecce-4f77-97eb-3f6bea0e5be4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:35.825146 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda64e2b-ecce-4f77-97eb-3f6bea0e5be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:35.825423 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda64e2b-ecce-4f77-97eb-3f6bea0e5be4 admin admin] PATH_INFO: `/v3/projects/c30c5e6090d34ef4bc14f047af450c6e/users/70bf791301ef47a4ba0eceeab850a88e/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:35.827482 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda64e2b-ecce-4f77-97eb-3f6bea0e5be4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c30c5e6090d34ef4bc14f047af450c6e, user_id=70bf791301ef47a4ba0eceeab850a88e, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:35.830114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda64e2b-ecce-4f77-97eb-3f6bea0e5be4 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:35.842003 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1439/2846] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:20:35 2022] PUT /identity/v3/projects/c30c5e6090d34ef4bc14f047af450c6e/users/70bf791301ef47a4ba0eceeab850a88e/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:35.853111 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b734e-e34d-4448-b1d9-e07e3b4e6e9f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:35.853411 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b734e-e34d-4448-b1d9-e07e3b4e6e9f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:35.853696 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b734e-e34d-4448-b1d9-e07e3b4e6e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:35.881623 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9d2b734e-e34d-4448-b1d9-e07e3b4e6e9f None None] MFA Rules not processed for user `70bf791301ef47a4ba0eceeab850a88e`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:35.882895 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9d2b734e-e34d-4448-b1d9-e07e3b4e6e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:35.953626 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1408/2847] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:20:35.967649 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda64e2b-ecce-4f77-97eb-3f6bea0e5be4 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:36.028711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda64e2b-ecce-4f77-97eb-3f6bea0e5be4 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:36.032566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884f6c8-3dab-4936-9fc0-deb21f084beb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:36.033227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884f6c8-3dab-4936-9fc0-deb21f084beb service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:36.033509 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884f6c8-3dab-4936-9fc0-deb21f084beb service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:36.033787 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884f6c8-3dab-4936-9fc0-deb21f084beb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:36.046987 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9884f6c8-3dab-4936-9fc0-deb21f084beb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:36.048290 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9884f6c8-3dab-4936-9fc0-deb21f084beb service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:36.059631 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1440/2848] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:20:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:20:36.466828 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b734e-e34d-4448-b1d9-e07e3b4e6e9f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:36.476792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:36.566946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b734e-e34d-4448-b1d9-e07e3b4e6e9f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:36.573188 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c8e17-927d-4eb3-97f0-43e3ac831258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:36.574782 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c8e17-927d-4eb3-97f0-43e3ac831258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:36.575306 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c8e17-927d-4eb3-97f0-43e3ac831258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:36.575946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c8e17-927d-4eb3-97f0-43e3ac831258 admin admin] PATH_INFO: `/v3/users/050b371b7ade41b2b58bdc4edf97ef6f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:36.577725 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444c8e17-927d-4eb3-97f0-43e3ac831258 admin admin] RBAC: Authorizing `identity:delete_user(user_id=050b371b7ade41b2b58bdc4edf97ef6f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:36.580277 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444c8e17-927d-4eb3-97f0-43e3ac831258 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:36.640168 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-444c8e17-927d-4eb3-97f0-43e3ac831258 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '050b371b7ade41b2b58bdc4edf97ef6f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:36.648739 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-444c8e17-927d-4eb3-97f0-43e3ac831258 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '050b371b7ade41b2b58bdc4edf97ef6f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:36.657615 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-444c8e17-927d-4eb3-97f0-43e3ac831258 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '050b371b7ade41b2b58bdc4edf97ef6f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:36.672435 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-444c8e17-927d-4eb3-97f0-43e3ac831258 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '050b371b7ade41b2b58bdc4edf97ef6f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:36.673233 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-444c8e17-927d-4eb3-97f0-43e3ac831258 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '050b371b7ade41b2b58bdc4edf97ef6f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:36.679844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1409/2849] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:20:36 2022] DELETE /identity/v3/users/050b371b7ade41b2b58bdc4edf97ef6f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:36.686368 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc2bc1-d65c-4bfe-80c8-991a18a6980a None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:36.686914 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc2bc1-d65c-4bfe-80c8-991a18a6980a None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:36.687224 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc2bc1-d65c-4bfe-80c8-991a18a6980a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:36.709878 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-07bc2bc1-d65c-4bfe-80c8-991a18a6980a None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:36.711200 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-07bc2bc1-d65c-4bfe-80c8-991a18a6980a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:36.777889 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1441/2850] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:20:36.788959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c8e17-927d-4eb3-97f0-43e3ac831258 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:36.790763 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-444c8e17-927d-4eb3-97f0-43e3ac831258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:36.864844 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c8e17-927d-4eb3-97f0-43e3ac831258 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:36.869143 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f3f45-897e-4b70-9e64-6dd27cfa9478 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:36.869902 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f3f45-897e-4b70-9e64-6dd27cfa9478 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:36.870233 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f3f45-897e-4b70-9e64-6dd27cfa9478 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:36.870560 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f3f45-897e-4b70-9e64-6dd27cfa9478 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:36.882384 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f3f45-897e-4b70-9e64-6dd27cfa9478 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:36.884186 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f3f45-897e-4b70-9e64-6dd27cfa9478 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:36.905928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1410/2851] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:20:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:20:37.229633 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc2bc1-d65c-4bfe-80c8-991a18a6980a None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:37.231687 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-07bc2bc1-d65c-4bfe-80c8-991a18a6980a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:37.251149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc2bc1-d65c-4bfe-80c8-991a18a6980a None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:37.256017 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb19b51-efce-4dbf-a1e8-755a9c8ca90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:37.256931 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb19b51-efce-4dbf-a1e8-755a9c8ca90d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:37.257204 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb19b51-efce-4dbf-a1e8-755a9c8ca90d admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:37.257479 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb19b51-efce-4dbf-a1e8-755a9c8ca90d admin admin] PATH_INFO: `/v3/projects/bd7690e7a8be4465b961ff5ae495a580` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:37.259372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb19b51-efce-4dbf-a1e8-755a9c8ca90d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd7690e7a8be4465b961ff5ae495a580)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:37.262088 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb19b51-efce-4dbf-a1e8-755a9c8ca90d admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:37.312095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:37.335954 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-afb19b51-efce-4dbf-a1e8-755a9c8ca90d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd7690e7a8be4465b961ff5ae495a580', 'request_id': 'req-afb19b51-efce-4dbf-a1e8-755a9c8ca90d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:37.344413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-afb19b51-efce-4dbf-a1e8-755a9c8ca90d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd7690e7a8be4465b961ff5ae495a580', 'request_id': 'req-afb19b51-efce-4dbf-a1e8-755a9c8ca90d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:37.353545 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-afb19b51-efce-4dbf-a1e8-755a9c8ca90d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd7690e7a8be4465b961ff5ae495a580', 'request_id': 'req-afb19b51-efce-4dbf-a1e8-755a9c8ca90d'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:37.370227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-afb19b51-efce-4dbf-a1e8-755a9c8ca90d admin admin] The token cache is being invalidate because project bd7690e7a8be4465b961ff5ae495a580 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:20:37.370500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-afb19b51-efce-4dbf-a1e8-755a9c8ca90d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:37.371255 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-afb19b51-efce-4dbf-a1e8-755a9c8ca90d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:37.372934 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1442/2852] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:20:37 2022] DELETE /identity/v3/projects/bd7690e7a8be4465b961ff5ae495a580 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:37.377399 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f3f45-897e-4b70-9e64-6dd27cfa9478 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:37.388631 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f3f45-897e-4b70-9e64-6dd27cfa9478 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:37.417793 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:37.461824 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b787b5-231f-4209-a815-bf82138fd309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:37.463547 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b787b5-231f-4209-a815-bf82138fd309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:37.464037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b787b5-231f-4209-a815-bf82138fd309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:37.464523 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b787b5-231f-4209-a815-bf82138fd309 admin admin] PATH_INFO: `/v3/users/70bf791301ef47a4ba0eceeab850a88e` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:37.466671 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b787b5-231f-4209-a815-bf82138fd309 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70bf791301ef47a4ba0eceeab850a88e)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:37.470227 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b787b5-231f-4209-a815-bf82138fd309 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:37.544646 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e0b787b5-231f-4209-a815-bf82138fd309 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70bf791301ef47a4ba0eceeab850a88e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:37.554090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e0b787b5-231f-4209-a815-bf82138fd309 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70bf791301ef47a4ba0eceeab850a88e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:37.566937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e0b787b5-231f-4209-a815-bf82138fd309 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70bf791301ef47a4ba0eceeab850a88e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:37.590593 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e0b787b5-231f-4209-a815-bf82138fd309 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70bf791301ef47a4ba0eceeab850a88e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:37.591891 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e0b787b5-231f-4209-a815-bf82138fd309 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70bf791301ef47a4ba0eceeab850a88e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:37.605093 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1411/2853] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:20:37 2022] DELETE /identity/v3/users/70bf791301ef47a4ba0eceeab850a88e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:37.962765 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb19b51-efce-4dbf-a1e8-755a9c8ca90d admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:37.964603 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-afb19b51-efce-4dbf-a1e8-755a9c8ca90d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:37.989162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb19b51-efce-4dbf-a1e8-755a9c8ca90d admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:38.074265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ad6f8-3c31-49ba-847c-1d3c69bfeb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:38.075183 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ad6f8-3c31-49ba-847c-1d3c69bfeb7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:38.075465 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ad6f8-3c31-49ba-847c-1d3c69bfeb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:38.075744 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ad6f8-3c31-49ba-847c-1d3c69bfeb7e admin admin] PATH_INFO: `/v3/projects/c30c5e6090d34ef4bc14f047af450c6e` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:38.076981 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ad6f8-3c31-49ba-847c-1d3c69bfeb7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c30c5e6090d34ef4bc14f047af450c6e)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:38.078974 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ad6f8-3c31-49ba-847c-1d3c69bfeb7e admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:38.146289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f07ad6f8-3c31-49ba-847c-1d3c69bfeb7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c30c5e6090d34ef4bc14f047af450c6e', 'request_id': 'req-f07ad6f8-3c31-49ba-847c-1d3c69bfeb7e'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:38.152869 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f07ad6f8-3c31-49ba-847c-1d3c69bfeb7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c30c5e6090d34ef4bc14f047af450c6e', 'request_id': 'req-f07ad6f8-3c31-49ba-847c-1d3c69bfeb7e'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:38.159727 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f07ad6f8-3c31-49ba-847c-1d3c69bfeb7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c30c5e6090d34ef4bc14f047af450c6e', 'request_id': 'req-f07ad6f8-3c31-49ba-847c-1d3c69bfeb7e'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:38.174621 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f07ad6f8-3c31-49ba-847c-1d3c69bfeb7e admin admin] The token cache is being invalidate because project c30c5e6090d34ef4bc14f047af450c6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:20:38.175023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f07ad6f8-3c31-49ba-847c-1d3c69bfeb7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:38.175802 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-f07ad6f8-3c31-49ba-847c-1d3c69bfeb7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:20:38.177535 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1443/2854] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:20:37 2022] DELETE /identity/v3/projects/c30c5e6090d34ef4bc14f047af450c6e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:38.182813 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0563f2-1d96-40b0-8b9d-cec696759450 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:38.183134 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0563f2-1d96-40b0-8b9d-cec696759450 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:38.183422 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0563f2-1d96-40b0-8b9d-cec696759450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:38.204081 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5c0563f2-1d96-40b0-8b9d-cec696759450 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:38.205344 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5c0563f2-1d96-40b0-8b9d-cec696759450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:38.275922 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1412/2855] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:20:38.284068 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c9bf8-a479-422c-9b04-bb537ad8ba1e None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:38.284396 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c9bf8-a479-422c-9b04-bb537ad8ba1e None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:38.284678 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c9bf8-a479-422c-9b04-bb537ad8ba1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:38.304671 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-7b9c9bf8-a479-422c-9b04-bb537ad8ba1e None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:38.306277 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7b9c9bf8-a479-422c-9b04-bb537ad8ba1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:38.315548 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1444/2856] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:20:38.321520 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0563f2-1d96-40b0-8b9d-cec696759450 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:38.339427 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0563f2-1d96-40b0-8b9d-cec696759450 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:38.346469 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779feb5-0c87-4683-90cd-eabca7b79a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:38.347231 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779feb5-0c87-4683-90cd-eabca7b79a3f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:38.347542 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779feb5-0c87-4683-90cd-eabca7b79a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:38.347834 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779feb5-0c87-4683-90cd-eabca7b79a3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:38.348573 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1779feb5-0c87-4683-90cd-eabca7b79a3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 15:20:38.348888 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1779feb5-0c87-4683-90cd-eabca7b79a3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:38.350958 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1779feb5-0c87-4683-90cd-eabca7b79a3f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:38.364495 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1413/2857] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 15:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 15:20:38.369222 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c9bf8-a479-422c-9b04-bb537ad8ba1e None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:38.379003 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c9bf8-a479-422c-9b04-bb537ad8ba1e None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:38.384547 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992e9ce-2107-4961-bd71-b82f5c0f9868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:38.385253 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992e9ce-2107-4961-bd71-b82f5c0f9868 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:38.385541 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992e9ce-2107-4961-bd71-b82f5c0f9868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:38.385819 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992e9ce-2107-4961-bd71-b82f5c0f9868 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:38.386571 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992e9ce-2107-4961-bd71-b82f5c0f9868 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:38.388543 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992e9ce-2107-4961-bd71-b82f5c0f9868 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:38.419997 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1445/2858] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 15:20:38 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 15:20:38.424777 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779feb5-0c87-4683-90cd-eabca7b79a3f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:38.435714 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779feb5-0c87-4683-90cd-eabca7b79a3f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:38.474045 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:38.514937 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b784c8-28ca-49bd-a3cc-f98784988206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:38.516139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b784c8-28ca-49bd-a3cc-f98784988206 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:38.516650 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b784c8-28ca-49bd-a3cc-f98784988206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:38.517163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b784c8-28ca-49bd-a3cc-f98784988206 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:38.518500 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b784c8-28ca-49bd-a3cc-f98784988206 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:38.521475 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b784c8-28ca-49bd-a3cc-f98784988206 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:38.546776 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1414/2859] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 15:20:38 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 15:20:38.551539 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992e9ce-2107-4961-bd71-b82f5c0f9868 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:38.567314 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992e9ce-2107-4961-bd71-b82f5c0f9868 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:38.573472 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc2391d-6495-4173-9a1d-be6e6764d0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:38.574173 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2391d-6495-4173-9a1d-be6e6764d0c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:38.574466 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2391d-6495-4173-9a1d-be6e6764d0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:38.574785 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2391d-6495-4173-9a1d-be6e6764d0c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:38.575575 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc2391d-6495-4173-9a1d-be6e6764d0c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:38.577273 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc2391d-6495-4173-9a1d-be6e6764d0c9 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:38.589662 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1446/2860] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:20:38.595739 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b784c8-28ca-49bd-a3cc-f98784988206 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:38.607377 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b784c8-28ca-49bd-a3cc-f98784988206 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:38.613396 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac284bc-f53a-4d0f-bf58-f693968a1df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:38.614123 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac284bc-f53a-4d0f-bf58-f693968a1df2 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:38.614430 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac284bc-f53a-4d0f-bf58-f693968a1df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:38.614727 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac284bc-f53a-4d0f-bf58-f693968a1df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:38.616947 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac284bc-f53a-4d0f-bf58-f693968a1df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:38.619484 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac284bc-f53a-4d0f-bf58-f693968a1df2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:38.635204 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1415/2861] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:20:38.639658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc2391d-6495-4173-9a1d-be6e6764d0c9 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:38.651215 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc2391d-6495-4173-9a1d-be6e6764d0c9 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:38.656517 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77090e0-4f16-4652-89e4-18e21810d645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:38.657433 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77090e0-4f16-4652-89e4-18e21810d645 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:38.657720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77090e0-4f16-4652-89e4-18e21810d645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:38.658014 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77090e0-4f16-4652-89e4-18e21810d645 admin admin] PATH_INFO: `/v3/projects/eec0132f71004ebab402589516b6a849/users/a07541bbaf1d4e8ea887269cfd7c5aa8/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:38.660621 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:38.680748 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77090e0-4f16-4652-89e4-18e21810d645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eec0132f71004ebab402589516b6a849, user_id=a07541bbaf1d4e8ea887269cfd7c5aa8, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:38.683414 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77090e0-4f16-4652-89e4-18e21810d645 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:38.702774 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1447/2862] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:20:38 2022] PUT /identity/v3/projects/eec0132f71004ebab402589516b6a849/users/a07541bbaf1d4e8ea887269cfd7c5aa8/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:38.708892 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b6ec9-947c-4570-9be3-3d20f198644c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:38.709175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b6ec9-947c-4570-9be3-3d20f198644c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:38.709458 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b6ec9-947c-4570-9be3-3d20f198644c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:38.746619 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1f5b6ec9-947c-4570-9be3-3d20f198644c None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:38.748965 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1f5b6ec9-947c-4570-9be3-3d20f198644c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:38.865359 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1416/2863] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 15:20:38.874259 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77090e0-4f16-4652-89e4-18e21810d645 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:38.876487 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b77090e0-4f16-4652-89e4-18e21810d645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:38.953049 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77090e0-4f16-4652-89e4-18e21810d645 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:38.953049 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a478f3-30c2-48a2-a0cb-99ca56291814 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:38.953049 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a478f3-30c2-48a2-a0cb-99ca56291814 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:38.953049 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a478f3-30c2-48a2-a0cb-99ca56291814 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:38.953767 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a478f3-30c2-48a2-a0cb-99ca56291814 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:38.964530 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a478f3-30c2-48a2-a0cb-99ca56291814 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:38.966179 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a478f3-30c2-48a2-a0cb-99ca56291814 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:38.978791 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1448/2864] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:20:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:20:39.827215 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256e535-b3d8-4b6c-b0bd-4579eec4981d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:39.827880 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256e535-b3d8-4b6c-b0bd-4579eec4981d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:39.828273 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256e535-b3d8-4b6c-b0bd-4579eec4981d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:39.881258 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7256e535-b3d8-4b6c-b0bd-4579eec4981d None None] MFA Rules not processed for user `a07541bbaf1d4e8ea887269cfd7c5aa8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:39.883936 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7256e535-b3d8-4b6c-b0bd-4579eec4981d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:40.027320 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1417/2865] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:20:40.031238 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a478f3-30c2-48a2-a0cb-99ca56291814 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:40.052269 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a478f3-30c2-48a2-a0cb-99ca56291814 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:40.070088 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69aa00f-b142-434b-85ee-9415cf867855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:40.071753 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69aa00f-b142-434b-85ee-9415cf867855 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:40.072551 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69aa00f-b142-434b-85ee-9415cf867855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:40.073286 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69aa00f-b142-434b-85ee-9415cf867855 admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:40.075020 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69aa00f-b142-434b-85ee-9415cf867855 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:40.079830 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69aa00f-b142-434b-85ee-9415cf867855 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:40.150749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1449/2866] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 15:20:40 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:20:40.153775 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256e535-b3d8-4b6c-b0bd-4579eec4981d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:40.172544 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256e535-b3d8-4b6c-b0bd-4579eec4981d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:40.228783 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:40.250089 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb50b32-4eb8-42ed-b867-64dad5617398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:40.250954 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb50b32-4eb8-42ed-b867-64dad5617398 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:40.251325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb50b32-4eb8-42ed-b867-64dad5617398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:40.251648 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb50b32-4eb8-42ed-b867-64dad5617398 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:40.252553 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb50b32-4eb8-42ed-b867-64dad5617398 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:40.254878 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb50b32-4eb8-42ed-b867-64dad5617398 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:40.280243 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1418/2867] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 15:20:40 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:20:40.284855 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69aa00f-b142-434b-85ee-9415cf867855 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:40.296551 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69aa00f-b142-434b-85ee-9415cf867855 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:40.303318 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba73bd0-8793-414d-ab9e-8f0bda6e5a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:40.304424 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba73bd0-8793-414d-ab9e-8f0bda6e5a23 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:40.305405 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba73bd0-8793-414d-ab9e-8f0bda6e5a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:40.305864 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba73bd0-8793-414d-ab9e-8f0bda6e5a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:40.307289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba73bd0-8793-414d-ab9e-8f0bda6e5a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:40.310077 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba73bd0-8793-414d-ab9e-8f0bda6e5a23 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:40.329386 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1450/2868] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:20:40.335562 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb50b32-4eb8-42ed-b867-64dad5617398 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:40.346848 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb50b32-4eb8-42ed-b867-64dad5617398 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:40.352086 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1506c64-99d9-430b-a19d-5aa2a7b18664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:40.353261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1506c64-99d9-430b-a19d-5aa2a7b18664 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:40.353604 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1506c64-99d9-430b-a19d-5aa2a7b18664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:40.353946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1506c64-99d9-430b-a19d-5aa2a7b18664 admin admin] PATH_INFO: `/v3/projects/06ff2348476a45e3a23c352b17cd0531/users/9efc89e8b141405c8a57daf5e75af3d6/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:40.372997 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1506c64-99d9-430b-a19d-5aa2a7b18664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06ff2348476a45e3a23c352b17cd0531, user_id=9efc89e8b141405c8a57daf5e75af3d6, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:40.375753 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1506c64-99d9-430b-a19d-5aa2a7b18664 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:40.393809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1419/2869] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:20:40 2022] PUT /identity/v3/projects/06ff2348476a45e3a23c352b17cd0531/users/9efc89e8b141405c8a57daf5e75af3d6/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:40.397973 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba73bd0-8793-414d-ab9e-8f0bda6e5a23 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:40.409650 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba73bd0-8793-414d-ab9e-8f0bda6e5a23 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:40.479691 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c911c3-8608-4589-ae03-ee236f58fc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:40.480487 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c911c3-8608-4589-ae03-ee236f58fc6a admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:40.480823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c911c3-8608-4589-ae03-ee236f58fc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:40.481162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c911c3-8608-4589-ae03-ee236f58fc6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:40.482062 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c911c3-8608-4589-ae03-ee236f58fc6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:40.483833 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c911c3-8608-4589-ae03-ee236f58fc6a admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:40.496362 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1451/2870] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 15:20:40.500724 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1506c64-99d9-430b-a19d-5aa2a7b18664 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:40.512491 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1506c64-99d9-430b-a19d-5aa2a7b18664 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:40.517601 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d923bb-d2d6-47af-b6ff-fe2137668573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:40.518531 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d923bb-d2d6-47af-b6ff-fe2137668573 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:40.518856 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d923bb-d2d6-47af-b6ff-fe2137668573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:40.519138 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d923bb-d2d6-47af-b6ff-fe2137668573 admin admin] PATH_INFO: `/v3/projects/06ff2348476a45e3a23c352b17cd0531/users/9efc89e8b141405c8a57daf5e75af3d6/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:40.521313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d923bb-d2d6-47af-b6ff-fe2137668573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06ff2348476a45e3a23c352b17cd0531, user_id=9efc89e8b141405c8a57daf5e75af3d6, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:40.524342 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d923bb-d2d6-47af-b6ff-fe2137668573 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:40.536903 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1420/2871] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:20:40 2022] PUT /identity/v3/projects/06ff2348476a45e3a23c352b17cd0531/users/9efc89e8b141405c8a57daf5e75af3d6/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:20:41.446827 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd388da-6375-4474-b78b-a99300ab079d None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:41.447823 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd388da-6375-4474-b78b-a99300ab079d None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:41.448278 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd388da-6375-4474-b78b-a99300ab079d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:41.495688 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-dfd388da-6375-4474-b78b-a99300ab079d None None] MFA Rules not processed for user `9efc89e8b141405c8a57daf5e75af3d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:20:41.497905 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-dfd388da-6375-4474-b78b-a99300ab079d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:41.565195 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:20:41.624972 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1452/2872] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:20:41 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:20:41.633797 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d923bb-d2d6-47af-b6ff-fe2137668573 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:41.636236 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-61d923bb-d2d6-47af-b6ff-fe2137668573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:41.713887 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d923bb-d2d6-47af-b6ff-fe2137668573 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:41.717893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24105a0-ef02-4202-b59d-860e8dfb5817 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:41.718583 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24105a0-ef02-4202-b59d-860e8dfb5817 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:41.718911 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24105a0-ef02-4202-b59d-860e8dfb5817 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:41.719196 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24105a0-ef02-4202-b59d-860e8dfb5817 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:41.794441 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24105a0-ef02-4202-b59d-860e8dfb5817 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:41.796694 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24105a0-ef02-4202-b59d-860e8dfb5817 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:41.808754 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1421/2873] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:20:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 15:20:41.902049 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd388da-6375-4474-b78b-a99300ab079d None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:41.904749 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-dfd388da-6375-4474-b78b-a99300ab079d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:41.918558 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd388da-6375-4474-b78b-a99300ab079d None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:41.924226 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc24c5-1a48-482e-b917-d0bbe29aacc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:41.925157 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc24c5-1a48-482e-b917-d0bbe29aacc1 service cinder] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:41.925515 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc24c5-1a48-482e-b917-d0bbe29aacc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:41.925895 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc24c5-1a48-482e-b917-d0bbe29aacc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:41.928475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7dcc24c5-1a48-482e-b917-d0bbe29aacc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:42.028774 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc24c5-1a48-482e-b917-d0bbe29aacc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:42.032200 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc24c5-1a48-482e-b917-d0bbe29aacc1 service cinder] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:42.054340 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1453/2874] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:20:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 15:20:43.319656 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24105a0-ef02-4202-b59d-860e8dfb5817 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:43.330037 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24105a0-ef02-4202-b59d-860e8dfb5817 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:43.334325 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12130b6d-0cea-4527-8c7c-d803cd92ff1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:43.335156 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12130b6d-0cea-4527-8c7c-d803cd92ff1e service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:43.335570 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12130b6d-0cea-4527-8c7c-d803cd92ff1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:43.336005 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12130b6d-0cea-4527-8c7c-d803cd92ff1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:43.347090 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12130b6d-0cea-4527-8c7c-d803cd92ff1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:43.348558 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12130b6d-0cea-4527-8c7c-d803cd92ff1e service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:43.360248 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1422/2875] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:20:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 31 15:20:50.305583 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc24c5-1a48-482e-b917-d0bbe29aacc1 service cinder] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:20:50.308105 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-7dcc24c5-1a48-482e-b917-d0bbe29aacc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:50.396944 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc24c5-1a48-482e-b917-d0bbe29aacc1 service cinder] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:20:50.403015 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c27efa-2c22-4bc5-b19e-5e4ac423f06d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:20:50.403873 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c27efa-2c22-4bc5-b19e-5e4ac423f06d service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:20:50.404244 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c27efa-2c22-4bc5-b19e-5e4ac423f06d service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:20:50.404726 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c27efa-2c22-4bc5-b19e-5e4ac423f06d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:20:50.407072 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f8c27efa-2c22-4bc5-b19e-5e4ac423f06d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:20:50.498682 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c27efa-2c22-4bc5-b19e-5e4ac423f06d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:20:50.501281 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c27efa-2c22-4bc5-b19e-5e4ac423f06d service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:20:50.522305 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1454/2876] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:20:50 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:20:53.354740 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=78,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:21:00.511091 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=92 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:21:12.019416 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12130b6d-0cea-4527-8c7c-d803cd92ff1e service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:12.093235 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12130b6d-0cea-4527-8c7c-d803cd92ff1e service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:12.100255 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d91a33-589a-4a4c-abaa-e98839cf0dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:12.102842 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d91a33-589a-4a4c-abaa-e98839cf0dba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:12.103191 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d91a33-589a-4a4c-abaa-e98839cf0dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:12.103552 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d91a33-589a-4a4c-abaa-e98839cf0dba admin admin] PATH_INFO: `/v3/users/a07541bbaf1d4e8ea887269cfd7c5aa8` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:12.105893 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d91a33-589a-4a4c-abaa-e98839cf0dba admin admin] RBAC: Authorizing `identity:delete_user(user_id=a07541bbaf1d4e8ea887269cfd7c5aa8)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:12.109785 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d91a33-589a-4a4c-abaa-e98839cf0dba admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:12.174024 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e7d91a33-589a-4a4c-abaa-e98839cf0dba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a07541bbaf1d4e8ea887269cfd7c5aa8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:12.183562 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e7d91a33-589a-4a4c-abaa-e98839cf0dba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a07541bbaf1d4e8ea887269cfd7c5aa8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:12.192808 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e7d91a33-589a-4a4c-abaa-e98839cf0dba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a07541bbaf1d4e8ea887269cfd7c5aa8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:12.209300 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e7d91a33-589a-4a4c-abaa-e98839cf0dba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a07541bbaf1d4e8ea887269cfd7c5aa8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:12.210284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e7d91a33-589a-4a4c-abaa-e98839cf0dba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a07541bbaf1d4e8ea887269cfd7c5aa8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:12.216514 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1423/2877] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:21:12 2022] DELETE /identity/v3/users/a07541bbaf1d4e8ea887269cfd7c5aa8 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:21:12.399870 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c27efa-2c22-4bc5-b19e-5e4ac423f06d service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:12.412541 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c27efa-2c22-4bc5-b19e-5e4ac423f06d service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:12.499765 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a59340-5325-4618-9404-78a93c024f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:12.500647 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a59340-5325-4618-9404-78a93c024f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:12.500935 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a59340-5325-4618-9404-78a93c024f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:12.501209 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a59340-5325-4618-9404-78a93c024f0b admin admin] PATH_INFO: `/v3/projects/eec0132f71004ebab402589516b6a849` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:12.502460 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a59340-5325-4618-9404-78a93c024f0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=eec0132f71004ebab402589516b6a849)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:12.504562 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a59340-5325-4618-9404-78a93c024f0b admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:12.557393 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a7a59340-5325-4618-9404-78a93c024f0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eec0132f71004ebab402589516b6a849', 'request_id': 'req-a7a59340-5325-4618-9404-78a93c024f0b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:12.564228 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a7a59340-5325-4618-9404-78a93c024f0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eec0132f71004ebab402589516b6a849', 'request_id': 'req-a7a59340-5325-4618-9404-78a93c024f0b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:12.571863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a7a59340-5325-4618-9404-78a93c024f0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eec0132f71004ebab402589516b6a849', 'request_id': 'req-a7a59340-5325-4618-9404-78a93c024f0b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:12.586309 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a7a59340-5325-4618-9404-78a93c024f0b admin admin] The token cache is being invalidate because project eec0132f71004ebab402589516b6a849 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:21:12.586601 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a7a59340-5325-4618-9404-78a93c024f0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:12.587271 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a7a59340-5325-4618-9404-78a93c024f0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:12.588892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1455/2878] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:21:12 2022] DELETE /identity/v3/projects/eec0132f71004ebab402589516b6a849 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 15:21:12.592964 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d91a33-589a-4a4c-abaa-e98839cf0dba admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:12.595006 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e7d91a33-589a-4a4c-abaa-e98839cf0dba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:12.613898 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d91a33-589a-4a4c-abaa-e98839cf0dba admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:12.675560 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c7698-1681-4a98-832a-48c9c80a11e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:12.676412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c7698-1681-4a98-832a-48c9c80a11e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:12.676699 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c7698-1681-4a98-832a-48c9c80a11e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:12.676979 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c7698-1681-4a98-832a-48c9c80a11e8 admin admin] PATH_INFO: `/v3/users/9efc89e8b141405c8a57daf5e75af3d6` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:12.678245 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c7698-1681-4a98-832a-48c9c80a11e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9efc89e8b141405c8a57daf5e75af3d6)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:12.680266 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c7698-1681-4a98-832a-48c9c80a11e8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:12.699094 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a59340-5325-4618-9404-78a93c024f0b admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:12.700649 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a7a59340-5325-4618-9404-78a93c024f0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:12.708679 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:21:12.715509 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a59340-5325-4618-9404-78a93c024f0b admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:12.722982 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c306e-aadf-4feb-8d54-58c7edd05154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:12.723892 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c306e-aadf-4feb-8d54-58c7edd05154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:12.724307 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c306e-aadf-4feb-8d54-58c7edd05154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:12.724752 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c306e-aadf-4feb-8d54-58c7edd05154 admin admin] PATH_INFO: `/v3/users/7e92cefa239743299a5bb1687dbc2994` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:12.726446 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c306e-aadf-4feb-8d54-58c7edd05154 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e92cefa239743299a5bb1687dbc2994)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:12.728449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c306e-aadf-4feb-8d54-58c7edd05154 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:12.740548 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6d6c7698-1681-4a98-832a-48c9c80a11e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9efc89e8b141405c8a57daf5e75af3d6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:12.748767 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6d6c7698-1681-4a98-832a-48c9c80a11e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9efc89e8b141405c8a57daf5e75af3d6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:12.757652 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6d6c7698-1681-4a98-832a-48c9c80a11e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9efc89e8b141405c8a57daf5e75af3d6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:12.771798 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6d6c7698-1681-4a98-832a-48c9c80a11e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9efc89e8b141405c8a57daf5e75af3d6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:12.773054 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6d6c7698-1681-4a98-832a-48c9c80a11e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9efc89e8b141405c8a57daf5e75af3d6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:12.779493 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1424/2879] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:21:12 2022] DELETE /identity/v3/users/9efc89e8b141405c8a57daf5e75af3d6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:21:12.783658 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-268c306e-aadf-4feb-8d54-58c7edd05154 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e92cefa239743299a5bb1687dbc2994'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:12.793378 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-268c306e-aadf-4feb-8d54-58c7edd05154 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e92cefa239743299a5bb1687dbc2994'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:12.794185 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-268c306e-aadf-4feb-8d54-58c7edd05154 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e92cefa239743299a5bb1687dbc2994'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:12.807838 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-268c306e-aadf-4feb-8d54-58c7edd05154 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e92cefa239743299a5bb1687dbc2994'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:12.808591 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-268c306e-aadf-4feb-8d54-58c7edd05154 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e92cefa239743299a5bb1687dbc2994'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:12.822085 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1456/2880] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:21:12 2022] DELETE /identity/v3/users/7e92cefa239743299a5bb1687dbc2994 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 15:21:12.826809 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b8533-3440-4bad-aaaf-beb8ca1f1c8b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:12.827118 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b8533-3440-4bad-aaaf-beb8ca1f1c8b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:12.827406 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b8533-3440-4bad-aaaf-beb8ca1f1c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:12.848205 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-268b8533-3440-4bad-aaaf-beb8ca1f1c8b None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:21:12.849464 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-268b8533-3440-4bad-aaaf-beb8ca1f1c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:12.920908 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1425/2881] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:21:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:21:12.931503 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c306e-aadf-4feb-8d54-58c7edd05154 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:12.935766 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-268c306e-aadf-4feb-8d54-58c7edd05154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:12.944745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:21:12.991753 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b8533-3440-4bad-aaaf-beb8ca1f1c8b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:12.993333 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-268b8533-3440-4bad-aaaf-beb8ca1f1c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:13.006367 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b8533-3440-4bad-aaaf-beb8ca1f1c8b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:13.018251 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d303eb6-83d2-48af-8054-ace5cc904f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:13.019237 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d303eb6-83d2-48af-8054-ace5cc904f5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:13.019532 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d303eb6-83d2-48af-8054-ace5cc904f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:13.020098 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d303eb6-83d2-48af-8054-ace5cc904f5f admin admin] PATH_INFO: `/v3/projects/06ff2348476a45e3a23c352b17cd0531` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:13.021581 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d303eb6-83d2-48af-8054-ace5cc904f5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=06ff2348476a45e3a23c352b17cd0531)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:13.024162 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d303eb6-83d2-48af-8054-ace5cc904f5f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:13.026510 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c306e-aadf-4feb-8d54-58c7edd05154 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:13.031556 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d658d-39f3-4262-b4af-d427cb238ba7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:13.032431 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d658d-39f3-4262-b4af-d427cb238ba7 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:13.032716 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d658d-39f3-4262-b4af-d427cb238ba7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:13.033001 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d658d-39f3-4262-b4af-d427cb238ba7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:13.043979 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d658d-39f3-4262-b4af-d427cb238ba7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:13.045103 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d658d-39f3-4262-b4af-d427cb238ba7 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:13.058114 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1457/2882] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:21:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:21:13.079890 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2d303eb6-83d2-48af-8054-ace5cc904f5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06ff2348476a45e3a23c352b17cd0531', 'request_id': 'req-2d303eb6-83d2-48af-8054-ace5cc904f5f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:13.087705 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2d303eb6-83d2-48af-8054-ace5cc904f5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06ff2348476a45e3a23c352b17cd0531', 'request_id': 'req-2d303eb6-83d2-48af-8054-ace5cc904f5f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:13.095807 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2d303eb6-83d2-48af-8054-ace5cc904f5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06ff2348476a45e3a23c352b17cd0531', 'request_id': 'req-2d303eb6-83d2-48af-8054-ace5cc904f5f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:13.112482 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2d303eb6-83d2-48af-8054-ace5cc904f5f admin admin] The token cache is being invalidate because project 06ff2348476a45e3a23c352b17cd0531 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:21:13.112813 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2d303eb6-83d2-48af-8054-ace5cc904f5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:13.113721 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2d303eb6-83d2-48af-8054-ace5cc904f5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:13.115520 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1426/2883] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:21:12 2022] DELETE /identity/v3/projects/06ff2348476a45e3a23c352b17cd0531 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:21:13.122076 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae728b-f876-4c75-9c5e-936fd360d627 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:13.122564 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae728b-f876-4c75-9c5e-936fd360d627 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:13.122871 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae728b-f876-4c75-9c5e-936fd360d627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:13.144069 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-64ae728b-f876-4c75-9c5e-936fd360d627 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:21:13.145499 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-64ae728b-f876-4c75-9c5e-936fd360d627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:13.223585 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1458/2884] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:21:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:21:13.236655 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308bc81-6446-4224-851b-2c82a2e0b879 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:13.238005 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308bc81-6446-4224-851b-2c82a2e0b879 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:13.238456 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308bc81-6446-4224-851b-2c82a2e0b879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:13.269217 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9308bc81-6446-4224-851b-2c82a2e0b879 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:21:13.270908 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9308bc81-6446-4224-851b-2c82a2e0b879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:13.282296 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1427/2885] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:21:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:21:13.287339 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae728b-f876-4c75-9c5e-936fd360d627 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:13.298717 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae728b-f876-4c75-9c5e-936fd360d627 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:13.305156 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200f48c-01af-46c6-8eed-8b0fe7e67672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:13.306470 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200f48c-01af-46c6-8eed-8b0fe7e67672 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:13.307904 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200f48c-01af-46c6-8eed-8b0fe7e67672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:13.308350 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200f48c-01af-46c6-8eed-8b0fe7e67672 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:13.309284 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200f48c-01af-46c6-8eed-8b0fe7e67672 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 15:21:13.310773 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200f48c-01af-46c6-8eed-8b0fe7e67672 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:13.312637 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200f48c-01af-46c6-8eed-8b0fe7e67672 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:13.331182 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1459/2886] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 15:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 15:21:13.332839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308bc81-6446-4224-851b-2c82a2e0b879 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:13.343565 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308bc81-6446-4224-851b-2c82a2e0b879 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:13.348517 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ff528-ab74-4748-b2ae-0b37deaf6150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:13.349302 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ff528-ab74-4748-b2ae-0b37deaf6150 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:13.349651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ff528-ab74-4748-b2ae-0b37deaf6150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:13.349996 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ff528-ab74-4748-b2ae-0b37deaf6150 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:13.351711 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ff528-ab74-4748-b2ae-0b37deaf6150 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:13.353253 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ff528-ab74-4748-b2ae-0b37deaf6150 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:13.368066 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:21:13.383512 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200f48c-01af-46c6-8eed-8b0fe7e67672 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:13.385456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b200f48c-01af-46c6-8eed-8b0fe7e67672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:13.388200 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1428/2887] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 15:21:13 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:21:13.392605 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ff528-ab74-4748-b2ae-0b37deaf6150 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:13.402025 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200f48c-01af-46c6-8eed-8b0fe7e67672 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:13.472492 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece6f60-f3f1-43f5-ab73-e11c268db59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:13.473291 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece6f60-f3f1-43f5-ab73-e11c268db59f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:13.473582 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece6f60-f3f1-43f5-ab73-e11c268db59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:13.473868 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece6f60-f3f1-43f5-ab73-e11c268db59f admin admin] PATH_INFO: `/v3/projects/e03724fb968b40808429b35a689bea3e` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:13.475151 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece6f60-f3f1-43f5-ab73-e11c268db59f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e03724fb968b40808429b35a689bea3e)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:13.477771 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece6f60-f3f1-43f5-ab73-e11c268db59f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:13.479206 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ff528-ab74-4748-b2ae-0b37deaf6150 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:13.484271 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599348d1-73dc-41fe-86af-3ce426b8e082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:13.484975 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599348d1-73dc-41fe-86af-3ce426b8e082 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:13.485266 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599348d1-73dc-41fe-86af-3ce426b8e082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:13.485554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599348d1-73dc-41fe-86af-3ce426b8e082 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:13.486261 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599348d1-73dc-41fe-86af-3ce426b8e082 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:13.488271 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599348d1-73dc-41fe-86af-3ce426b8e082 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:13.511892 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1429/2888] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 15:21:13 2022] POST /identity/v3/users => generated 290 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:21:13.516423 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599348d1-73dc-41fe-86af-3ce426b8e082 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:13.527697 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:21:13.528163 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599348d1-73dc-41fe-86af-3ce426b8e082 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:13.548103 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-aece6f60-f3f1-43f5-ab73-e11c268db59f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e03724fb968b40808429b35a689bea3e', 'request_id': 'req-aece6f60-f3f1-43f5-ab73-e11c268db59f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:13.556254 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-aece6f60-f3f1-43f5-ab73-e11c268db59f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e03724fb968b40808429b35a689bea3e', 'request_id': 'req-aece6f60-f3f1-43f5-ab73-e11c268db59f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:13.563622 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-aece6f60-f3f1-43f5-ab73-e11c268db59f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e03724fb968b40808429b35a689bea3e', 'request_id': 'req-aece6f60-f3f1-43f5-ab73-e11c268db59f'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:13.582342 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-aece6f60-f3f1-43f5-ab73-e11c268db59f admin admin] The token cache is being invalidate because project e03724fb968b40808429b35a689bea3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:21:13.582687 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-aece6f60-f3f1-43f5-ab73-e11c268db59f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:13.583439 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-aece6f60-f3f1-43f5-ab73-e11c268db59f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:13.585656 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1460/2889] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:21:13 2022] DELETE /identity/v3/projects/e03724fb968b40808429b35a689bea3e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:21:13.589580 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece6f60-f3f1-43f5-ab73-e11c268db59f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:13.591505 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-aece6f60-f3f1-43f5-ab73-e11c268db59f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:13.611114 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06d7f6-a063-48e0-9d02-1ee01c26d916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:13.612518 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06d7f6-a063-48e0-9d02-1ee01c26d916 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:13.613027 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06d7f6-a063-48e0-9d02-1ee01c26d916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:13.613510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06d7f6-a063-48e0-9d02-1ee01c26d916 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:13.614972 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06d7f6-a063-48e0-9d02-1ee01c26d916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:13.617946 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06d7f6-a063-48e0-9d02-1ee01c26d916 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:13.633321 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1430/2890] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:21:13.637731 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06d7f6-a063-48e0-9d02-1ee01c26d916 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:13.639907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cf06d7f6-a063-48e0-9d02-1ee01c26d916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:13.655570 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06d7f6-a063-48e0-9d02-1ee01c26d916 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:13.661725 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecebeb-b416-4074-bb3e-5727782e2dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:13.663418 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecebeb-b416-4074-bb3e-5727782e2dcf admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:13.663638 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece6f60-f3f1-43f5-ab73-e11c268db59f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:13.663758 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecebeb-b416-4074-bb3e-5727782e2dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:13.664044 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecebeb-b416-4074-bb3e-5727782e2dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:13.665313 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ecebeb-b416-4074-bb3e-5727782e2dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:13.668834 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59e3ed-456c-4928-a389-30a15c5baec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:13.669776 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59e3ed-456c-4928-a389-30a15c5baec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:13.670159 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59e3ed-456c-4928-a389-30a15c5baec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:13.670440 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59e3ed-456c-4928-a389-30a15c5baec0 admin admin] PATH_INFO: `/v3/users/bcab8a621e0c498a89e1c8f006789d39` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:13.671740 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b59e3ed-456c-4928-a389-30a15c5baec0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcab8a621e0c498a89e1c8f006789d39)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:13.673873 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b59e3ed-456c-4928-a389-30a15c5baec0 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:13.675228 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ecebeb-b416-4074-bb3e-5727782e2dcf admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:13.686295 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1431/2891] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 15:21:13.690381 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecebeb-b416-4074-bb3e-5727782e2dcf admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:13.702277 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecebeb-b416-4074-bb3e-5727782e2dcf admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:13.712323 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36691f3d-061e-45b2-806b-a3849dcf079c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:13.713240 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36691f3d-061e-45b2-806b-a3849dcf079c admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:13.713533 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36691f3d-061e-45b2-806b-a3849dcf079c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:13.713813 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36691f3d-061e-45b2-806b-a3849dcf079c admin admin] PATH_INFO: `/v3/projects/2dac67cb14e24c41b4add3b40a103fea/users/65894910ee8647f7b32395c0017d3562/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:13.726372 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1b59e3ed-456c-4928-a389-30a15c5baec0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcab8a621e0c498a89e1c8f006789d39'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:13.731148 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36691f3d-061e-45b2-806b-a3849dcf079c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dac67cb14e24c41b4add3b40a103fea, user_id=65894910ee8647f7b32395c0017d3562, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:13.733873 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36691f3d-061e-45b2-806b-a3849dcf079c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:13.736493 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1b59e3ed-456c-4928-a389-30a15c5baec0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcab8a621e0c498a89e1c8f006789d39'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:13.737545 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1b59e3ed-456c-4928-a389-30a15c5baec0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcab8a621e0c498a89e1c8f006789d39'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:13.751680 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1b59e3ed-456c-4928-a389-30a15c5baec0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcab8a621e0c498a89e1c8f006789d39'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:13.752083 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1432/2892] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:21:13 2022] PUT /identity/v3/projects/2dac67cb14e24c41b4add3b40a103fea/users/65894910ee8647f7b32395c0017d3562/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 15:21:13.752483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-1b59e3ed-456c-4928-a389-30a15c5baec0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcab8a621e0c498a89e1c8f006789d39'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:13.760435 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b60da-0362-487f-a272-440af49fbcd8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:13.760940 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b60da-0362-487f-a272-440af49fbcd8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:13.761428 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b60da-0362-487f-a272-440af49fbcd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:13.783856 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1461/2893] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:21:13 2022] DELETE /identity/v3/users/bcab8a621e0c498a89e1c8f006789d39 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 15:21:13.785380 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3f1b60da-0362-487f-a272-440af49fbcd8 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:21:13.786909 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3f1b60da-0362-487f-a272-440af49fbcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:13.830761 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:21:13.854906 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1433/2894] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:21:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:21:13.866859 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59e3ed-456c-4928-a389-30a15c5baec0 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:13.868451 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-1b59e3ed-456c-4928-a389-30a15c5baec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:13.921962 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:21:13.942325 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59e3ed-456c-4928-a389-30a15c5baec0 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:13.947996 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d828a-391d-4ba6-8e9e-7e944bf44b0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:13.948975 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d828a-391d-4ba6-8e9e-7e944bf44b0c service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:13.949391 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d828a-391d-4ba6-8e9e-7e944bf44b0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:13.949846 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d828a-391d-4ba6-8e9e-7e944bf44b0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:13.965488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d828a-391d-4ba6-8e9e-7e944bf44b0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:13.967056 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d828a-391d-4ba6-8e9e-7e944bf44b0c service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:13.978714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1462/2895] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:21:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:21:14.157821 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b60da-0362-487f-a272-440af49fbcd8 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:14.160091 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3f1b60da-0362-487f-a272-440af49fbcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:14.188663 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b60da-0362-487f-a272-440af49fbcd8 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:14.198125 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12902bb3-50e8-45a7-aa3d-10f74aa72565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:14.200554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12902bb3-50e8-45a7-aa3d-10f74aa72565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:14.201081 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12902bb3-50e8-45a7-aa3d-10f74aa72565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:14.201597 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12902bb3-50e8-45a7-aa3d-10f74aa72565 admin admin] PATH_INFO: `/v3/projects/44736e52b3984b838a71040d4f5f8352` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:14.205005 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12902bb3-50e8-45a7-aa3d-10f74aa72565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44736e52b3984b838a71040d4f5f8352)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:14.208743 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12902bb3-50e8-45a7-aa3d-10f74aa72565 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:14.279531 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-12902bb3-50e8-45a7-aa3d-10f74aa72565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44736e52b3984b838a71040d4f5f8352', 'request_id': 'req-12902bb3-50e8-45a7-aa3d-10f74aa72565'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:14.286442 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-12902bb3-50e8-45a7-aa3d-10f74aa72565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44736e52b3984b838a71040d4f5f8352', 'request_id': 'req-12902bb3-50e8-45a7-aa3d-10f74aa72565'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:14.293279 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-12902bb3-50e8-45a7-aa3d-10f74aa72565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44736e52b3984b838a71040d4f5f8352', 'request_id': 'req-12902bb3-50e8-45a7-aa3d-10f74aa72565'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:14.315124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-12902bb3-50e8-45a7-aa3d-10f74aa72565 admin admin] The token cache is being invalidate because project 44736e52b3984b838a71040d4f5f8352 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:21:14.315124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-12902bb3-50e8-45a7-aa3d-10f74aa72565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:14.315124 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-12902bb3-50e8-45a7-aa3d-10f74aa72565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:14.316102 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1434/2896] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:21:14 2022] DELETE /identity/v3/projects/44736e52b3984b838a71040d4f5f8352 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:21:14.324024 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44823e-faea-4f30-b9d0-c507850fd4be None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:14.324369 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44823e-faea-4f30-b9d0-c507850fd4be None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:14.324729 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44823e-faea-4f30-b9d0-c507850fd4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:14.344999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-3b44823e-faea-4f30-b9d0-c507850fd4be None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:21:14.346445 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-3b44823e-faea-4f30-b9d0-c507850fd4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:14.440921 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1463/2897] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:21:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:21:14.451910 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19b45c-0a83-4720-b0c6-f8ba6848d795 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:14.452406 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19b45c-0a83-4720-b0c6-f8ba6848d795 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:14.452825 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19b45c-0a83-4720-b0c6-f8ba6848d795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:14.492299 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ad19b45c-0a83-4720-b0c6-f8ba6848d795 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:21:14.493701 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ad19b45c-0a83-4720-b0c6-f8ba6848d795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:14.503744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1435/2898] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:21:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:21:14.508304 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44823e-faea-4f30-b9d0-c507850fd4be None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:14.518483 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44823e-faea-4f30-b9d0-c507850fd4be None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:14.523514 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886fec87-363a-427a-a2f8-9e7c4022c5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:14.524205 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fec87-363a-427a-a2f8-9e7c4022c5a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:14.524488 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fec87-363a-427a-a2f8-9e7c4022c5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:14.524766 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fec87-363a-427a-a2f8-9e7c4022c5a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:14.525506 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886fec87-363a-427a-a2f8-9e7c4022c5a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 15:21:14.526062 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886fec87-363a-427a-a2f8-9e7c4022c5a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:14.527871 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886fec87-363a-427a-a2f8-9e7c4022c5a5 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:14.543839 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1464/2899] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 15:21:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 15:21:14.548284 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19b45c-0a83-4720-b0c6-f8ba6848d795 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:14.563907 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19b45c-0a83-4720-b0c6-f8ba6848d795 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:14.569936 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ceb27-6593-4aa3-a578-d8fe73ff9655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:14.570639 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ceb27-6593-4aa3-a578-d8fe73ff9655 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:14.571106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ceb27-6593-4aa3-a578-d8fe73ff9655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:14.571393 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ceb27-6593-4aa3-a578-d8fe73ff9655 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:14.572106 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ceb27-6593-4aa3-a578-d8fe73ff9655 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:14.574053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ceb27-6593-4aa3-a578-d8fe73ff9655 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:14.609244 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1436/2900] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 15:21:14 2022] POST /identity/v3/projects => generated 387 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:21:14.614849 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886fec87-363a-427a-a2f8-9e7c4022c5a5 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:14.625814 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886fec87-363a-427a-a2f8-9e7c4022c5a5 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:14.710057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b79aa-4fd7-406d-b3d1-cc629ca7a735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:14.710057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b79aa-4fd7-406d-b3d1-cc629ca7a735 admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:14.710057 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b79aa-4fd7-406d-b3d1-cc629ca7a735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:14.710788 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b79aa-4fd7-406d-b3d1-cc629ca7a735 admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:14.711266 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b79aa-4fd7-406d-b3d1-cc629ca7a735 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:14.713306 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b79aa-4fd7-406d-b3d1-cc629ca7a735 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:14.738781 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1465/2901] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 15:21:14 2022] POST /identity/v3/users => generated 294 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 15:21:14.743330 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ceb27-6593-4aa3-a578-d8fe73ff9655 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:14.766139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ceb27-6593-4aa3-a578-d8fe73ff9655 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:14.774927 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736bbb5-24ba-469b-8341-55ebb64ce1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:14.776440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736bbb5-24ba-469b-8341-55ebb64ce1ba admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:14.776700 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736bbb5-24ba-469b-8341-55ebb64ce1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:14.777346 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736bbb5-24ba-469b-8341-55ebb64ce1ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:14.778212 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736bbb5-24ba-469b-8341-55ebb64ce1ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:14.781595 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736bbb5-24ba-469b-8341-55ebb64ce1ba admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:14.802066 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1437/2902] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:21:14.803782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b79aa-4fd7-406d-b3d1-cc629ca7a735 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:14.805841 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23795e5e-4730-4d75-9eee-740a85784a2a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:14.806130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23795e5e-4730-4d75-9eee-740a85784a2a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:14.806434 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23795e5e-4730-4d75-9eee-740a85784a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:14.818295 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b79aa-4fd7-406d-b3d1-cc629ca7a735 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:14.824178 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfec66a-11c1-47be-8d05-e29f3f15d362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:14.825096 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfec66a-11c1-47be-8d05-e29f3f15d362 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:14.825453 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfec66a-11c1-47be-8d05-e29f3f15d362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:14.825827 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfec66a-11c1-47be-8d05-e29f3f15d362 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:14.826688 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfec66a-11c1-47be-8d05-e29f3f15d362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:14.828403 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfec66a-11c1-47be-8d05-e29f3f15d362 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:14.840461 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1466/2903] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 31 15:21:14.843950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-23795e5e-4730-4d75-9eee-740a85784a2a None None] MFA Rules not processed for user `65894910ee8647f7b32395c0017d3562`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:21:14.845637 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-23795e5e-4730-4d75-9eee-740a85784a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:14.846115 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfec66a-11c1-47be-8d05-e29f3f15d362 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:14.865229 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfec66a-11c1-47be-8d05-e29f3f15d362 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:14.873848 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f18a6-51e8-47bf-9590-7cce58cbcf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:14.874883 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f18a6-51e8-47bf-9590-7cce58cbcf67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:14.875427 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f18a6-51e8-47bf-9590-7cce58cbcf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:14.875720 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f18a6-51e8-47bf-9590-7cce58cbcf67 admin admin] PATH_INFO: `/v3/projects/7722c8af92c047f585e425b430cd1382/users/74bc2d7ac4524e7bb2a5edc4d9daee36/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:14.895718 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f18a6-51e8-47bf-9590-7cce58cbcf67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7722c8af92c047f585e425b430cd1382, user_id=74bc2d7ac4524e7bb2a5edc4d9daee36, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:14.899560 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f18a6-51e8-47bf-9590-7cce58cbcf67 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:14.918529 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1467/2904] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:21:14 2022] PUT /identity/v3/projects/7722c8af92c047f585e425b430cd1382/users/74bc2d7ac4524e7bb2a5edc4d9daee36/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 15:21:14.934500 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cb818-a7f7-4df0-883f-74d147868069 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:14.934943 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cb818-a7f7-4df0-883f-74d147868069 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:14.935162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cb818-a7f7-4df0-883f-74d147868069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:14.955014 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1438/2905] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:21:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:21:14.969792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23795e5e-4730-4d75-9eee-740a85784a2a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:14.971640 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-23795e5e-4730-4d75-9eee-740a85784a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:14.972989 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-f75cb818-a7f7-4df0-883f-74d147868069 None None] MFA Rules not processed for user `74bc2d7ac4524e7bb2a5edc4d9daee36`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:21:14.974456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-f75cb818-a7f7-4df0-883f-74d147868069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:14.994343 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:21:14.995948 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:21:15.051151 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23795e5e-4730-4d75-9eee-740a85784a2a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:15.080607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1468/2906] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:21:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 15:21:15.086135 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75cb818-a7f7-4df0-883f-74d147868069 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:15.112033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129b6b2-5185-4035-b8f1-81d68f6b784d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:15.112716 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129b6b2-5185-4035-b8f1-81d68f6b784d service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:15.113002 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129b6b2-5185-4035-b8f1-81d68f6b784d service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:15.113275 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129b6b2-5185-4035-b8f1-81d68f6b784d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:15.160566 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75cb818-a7f7-4df0-883f-74d147868069 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:15.165741 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e49b4-91d7-44b6-a21a-9c53ba01979f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:15.166434 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e49b4-91d7-44b6-a21a-9c53ba01979f admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:15.166744 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e49b4-91d7-44b6-a21a-9c53ba01979f admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:15.167027 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e49b4-91d7-44b6-a21a-9c53ba01979f admin admin] PATH_INFO: `/v3/projects` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:15.167724 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e49b4-91d7-44b6-a21a-9c53ba01979f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:15.169653 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e49b4-91d7-44b6-a21a-9c53ba01979f admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:15.175801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129b6b2-5185-4035-b8f1-81d68f6b784d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:15.177247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129b6b2-5185-4035-b8f1-81d68f6b784d service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:15.187631 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1439/2907] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:21:14 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:21:15.197884 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1469/2908] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 15:21:15 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 15:21:15.202229 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129b6b2-5185-4035-b8f1-81d68f6b784d service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:15.217828 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129b6b2-5185-4035-b8f1-81d68f6b784d service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:15.289322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c49133-1d9b-4302-9688-14fc7914af3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:15.290109 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c49133-1d9b-4302-9688-14fc7914af3d admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:15.290343 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c49133-1d9b-4302-9688-14fc7914af3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:15.290673 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c49133-1d9b-4302-9688-14fc7914af3d admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:15.291700 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c49133-1d9b-4302-9688-14fc7914af3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:15.293793 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c49133-1d9b-4302-9688-14fc7914af3d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:15.316883 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1440/2909] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 15:21:15 2022] POST /identity/v3/users => generated 300 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 31 15:21:15.322600 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e49b4-91d7-44b6-a21a-9c53ba01979f admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:15.338149 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e49b4-91d7-44b6-a21a-9c53ba01979f admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:15.344352 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15af5e-ee03-4302-928e-8178876fd92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:15.345162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15af5e-ee03-4302-928e-8178876fd92c admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:15.345449 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15af5e-ee03-4302-928e-8178876fd92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:15.345732 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15af5e-ee03-4302-928e-8178876fd92c admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:15.346512 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15af5e-ee03-4302-928e-8178876fd92c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:15.348526 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15af5e-ee03-4302-928e-8178876fd92c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:15.360786 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1470/2910] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:21:15.365736 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c49133-1d9b-4302-9688-14fc7914af3d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:15.377094 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c49133-1d9b-4302-9688-14fc7914af3d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:15.384006 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277d8ef-dba6-48de-afe6-fa5c542c860e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:15.384942 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277d8ef-dba6-48de-afe6-fa5c542c860e admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:15.385323 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277d8ef-dba6-48de-afe6-fa5c542c860e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:15.385706 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277d8ef-dba6-48de-afe6-fa5c542c860e admin admin] PATH_INFO: `/v3/projects/c35d3a62e24d40eb923d082a8b4ae874/users/5b0cf2c787e540b4ba4cd27ec04bbd20/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:15.405098 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2277d8ef-dba6-48de-afe6-fa5c542c860e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c35d3a62e24d40eb923d082a8b4ae874, user_id=5b0cf2c787e540b4ba4cd27ec04bbd20, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:15.407800 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2277d8ef-dba6-48de-afe6-fa5c542c860e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:15.410336 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:21:15.427243 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1441/2911] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:21:15 2022] PUT /identity/v3/projects/c35d3a62e24d40eb923d082a8b4ae874/users/5b0cf2c787e540b4ba4cd27ec04bbd20/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:21:15.431360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15af5e-ee03-4302-928e-8178876fd92c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:15.442017 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15af5e-ee03-4302-928e-8178876fd92c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:15.532767 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220feec7-6557-419c-847b-960c3fb0f932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:15.533778 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220feec7-6557-419c-847b-960c3fb0f932 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:15.534094 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220feec7-6557-419c-847b-960c3fb0f932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:15.534477 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220feec7-6557-419c-847b-960c3fb0f932 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:15.535388 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220feec7-6557-419c-847b-960c3fb0f932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:15.537138 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220feec7-6557-419c-847b-960c3fb0f932 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:15.554596 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1471/2912] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:21:15.558676 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277d8ef-dba6-48de-afe6-fa5c542c860e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:15.572056 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277d8ef-dba6-48de-afe6-fa5c542c860e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:15.582761 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9711e25-04f9-4e18-afd5-dcd255aaa428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:15.583934 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9711e25-04f9-4e18-afd5-dcd255aaa428 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:15.584231 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9711e25-04f9-4e18-afd5-dcd255aaa428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:15.584513 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9711e25-04f9-4e18-afd5-dcd255aaa428 admin admin] PATH_INFO: `/v3/projects/c35d3a62e24d40eb923d082a8b4ae874/users/5b0cf2c787e540b4ba4cd27ec04bbd20/roles/e8d679596ffb45939229d1e2a13ce9b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:15.587126 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9711e25-04f9-4e18-afd5-dcd255aaa428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c35d3a62e24d40eb923d082a8b4ae874, user_id=5b0cf2c787e540b4ba4cd27ec04bbd20, role_id=e8d679596ffb45939229d1e2a13ce9b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:15.589898 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9711e25-04f9-4e18-afd5-dcd255aaa428 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:15.602746 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1442/2913] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:21:15 2022] PUT /identity/v3/projects/c35d3a62e24d40eb923d082a8b4ae874/users/5b0cf2c787e540b4ba4cd27ec04bbd20/roles/e8d679596ffb45939229d1e2a13ce9b0 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 15:21:15.613279 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb48fa-5e3b-4ea4-9c93-e404e8435d2f None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:15.613615 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb48fa-5e3b-4ea4-9c93-e404e8435d2f None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:15.613849 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb48fa-5e3b-4ea4-9c93-e404e8435d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:15.644537 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-01fb48fa-5e3b-4ea4-9c93-e404e8435d2f None None] MFA Rules not processed for user `5b0cf2c787e540b4ba4cd27ec04bbd20`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:21:15.645808 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-01fb48fa-5e3b-4ea4-9c93-e404e8435d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:15.659623 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:21:15.856723 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1472/2914] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:21:15 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:21:15.869769 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9711e25-04f9-4e18-afd5-dcd255aaa428 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:15.927576 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9711e25-04f9-4e18-afd5-dcd255aaa428 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:15.931748 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55dbad-48fe-47e7-bcb9-3a0dd5b994ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990999672d1a4f19896a143e1eeddd44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:15.932409 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55dbad-48fe-47e7-bcb9-3a0dd5b994ef service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:15.932690 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55dbad-48fe-47e7-bcb9-3a0dd5b994ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:15.932963 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55dbad-48fe-47e7-bcb9-3a0dd5b994ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:15.943940 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b55dbad-48fe-47e7-bcb9-3a0dd5b994ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:15.944928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b55dbad-48fe-47e7-bcb9-3a0dd5b994ef service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:15.955908 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1443/2915] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:21:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:21:15.987434 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb48fa-5e3b-4ea4-9c93-e404e8435d2f None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:16.059591 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb48fa-5e3b-4ea4-9c93-e404e8435d2f None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:16.064145 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c86e94-b139-42f8-94e4-95e2668c9180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:16.065029 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c86e94-b139-42f8-94e4-95e2668c9180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:16.065303 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c86e94-b139-42f8-94e4-95e2668c9180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:16.065598 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c86e94-b139-42f8-94e4-95e2668c9180 admin admin] PATH_INFO: `/v3/users/74bc2d7ac4524e7bb2a5edc4d9daee36` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:16.066863 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c86e94-b139-42f8-94e4-95e2668c9180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74bc2d7ac4524e7bb2a5edc4d9daee36)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:16.068782 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c86e94-b139-42f8-94e4-95e2668c9180 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:16.118701 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-26c86e94-b139-42f8-94e4-95e2668c9180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74bc2d7ac4524e7bb2a5edc4d9daee36'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:16.127715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-26c86e94-b139-42f8-94e4-95e2668c9180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74bc2d7ac4524e7bb2a5edc4d9daee36'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:16.128350 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-26c86e94-b139-42f8-94e4-95e2668c9180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74bc2d7ac4524e7bb2a5edc4d9daee36'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:16.142484 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-26c86e94-b139-42f8-94e4-95e2668c9180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74bc2d7ac4524e7bb2a5edc4d9daee36'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:16.143282 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-26c86e94-b139-42f8-94e4-95e2668c9180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74bc2d7ac4524e7bb2a5edc4d9daee36'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:16.156466 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1473/2916] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:21:15 2022] DELETE /identity/v3/users/74bc2d7ac4524e7bb2a5edc4d9daee36 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:21:16.161013 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910db58-6282-441d-a3e0-37c05e2ecd28 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:16.161301 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910db58-6282-441d-a3e0-37c05e2ecd28 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:16.161583 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910db58-6282-441d-a3e0-37c05e2ecd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:16.180462 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c910db58-6282-441d-a3e0-37c05e2ecd28 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:21:16.181746 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c910db58-6282-441d-a3e0-37c05e2ecd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:16.256260 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1444/2917] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:21:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:21:16.263442 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c86e94-b139-42f8-94e4-95e2668c9180 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:16.268236 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-26c86e94-b139-42f8-94e4-95e2668c9180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:16.323609 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:21:16.352802 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c86e94-b139-42f8-94e4-95e2668c9180 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:16.357177 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bf0a24-6fdc-4cc3-b887-adf6a8af6e05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:16.357898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf0a24-6fdc-4cc3-b887-adf6a8af6e05 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:16.358185 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf0a24-6fdc-4cc3-b887-adf6a8af6e05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:16.358467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf0a24-6fdc-4cc3-b887-adf6a8af6e05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:16.369710 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bf0a24-6fdc-4cc3-b887-adf6a8af6e05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:16.371129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bf0a24-6fdc-4cc3-b887-adf6a8af6e05 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:16.381870 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1474/2918] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:21:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:21:16.651515 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e60299-58c2-484d-b7fb-0d022931c50d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:16.651811 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e60299-58c2-484d-b7fb-0d022931c50d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:16.652094 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e60299-58c2-484d-b7fb-0d022931c50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:16.670064 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d5e60299-58c2-484d-b7fb-0d022931c50d None None] MFA Rules not processed for user `65894910ee8647f7b32395c0017d3562`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:21:16.671322 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d5e60299-58c2-484d-b7fb-0d022931c50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:16.676932 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bf0a24-6fdc-4cc3-b887-adf6a8af6e05 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:16.678991 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-98bf0a24-6fdc-4cc3-b887-adf6a8af6e05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:16.692864 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bf0a24-6fdc-4cc3-b887-adf6a8af6e05 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:16.698278 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2a0e3-5527-4941-b85f-0c1776e28caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:16.699264 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2a0e3-5527-4941-b85f-0c1776e28caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:16.699543 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2a0e3-5527-4941-b85f-0c1776e28caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:16.699825 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2a0e3-5527-4941-b85f-0c1776e28caf admin admin] PATH_INFO: `/v3/projects/7722c8af92c047f585e425b430cd1382` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:16.701061 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b2a0e3-5527-4941-b85f-0c1776e28caf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7722c8af92c047f585e425b430cd1382)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:16.703475 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b2a0e3-5527-4941-b85f-0c1776e28caf admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:16.736440 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1445/2919] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:21:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:21:16.753053 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b5b2a0e3-5527-4941-b85f-0c1776e28caf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7722c8af92c047f585e425b430cd1382', 'request_id': 'req-b5b2a0e3-5527-4941-b85f-0c1776e28caf'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:16.761155 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b5b2a0e3-5527-4941-b85f-0c1776e28caf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7722c8af92c047f585e425b430cd1382', 'request_id': 'req-b5b2a0e3-5527-4941-b85f-0c1776e28caf'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:16.767687 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b5b2a0e3-5527-4941-b85f-0c1776e28caf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7722c8af92c047f585e425b430cd1382', 'request_id': 'req-b5b2a0e3-5527-4941-b85f-0c1776e28caf'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:16.785265 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b5b2a0e3-5527-4941-b85f-0c1776e28caf admin admin] The token cache is being invalidate because project 7722c8af92c047f585e425b430cd1382 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:21:16.785736 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b5b2a0e3-5527-4941-b85f-0c1776e28caf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:16.786718 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-b5b2a0e3-5527-4941-b85f-0c1776e28caf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:16.789360 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1475/2920] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:21:16 2022] DELETE /identity/v3/projects/7722c8af92c047f585e425b430cd1382 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 31 15:21:16.793624 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e60299-58c2-484d-b7fb-0d022931c50d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:16.805579 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e60299-58c2-484d-b7fb-0d022931c50d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:16.832651 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:21:16.869881 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1ef97-756a-4891-b15d-fa5c09d0fc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:16.870801 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1ef97-756a-4891-b15d-fa5c09d0fc85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:16.871140 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1ef97-756a-4891-b15d-fa5c09d0fc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:16.871368 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1ef97-756a-4891-b15d-fa5c09d0fc85 admin admin] PATH_INFO: `/v3/users/5b0cf2c787e540b4ba4cd27ec04bbd20` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:16.872583 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1ef97-756a-4891-b15d-fa5c09d0fc85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b0cf2c787e540b4ba4cd27ec04bbd20)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:16.874524 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1ef97-756a-4891-b15d-fa5c09d0fc85 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:16.925128 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dac1ef97-756a-4891-b15d-fa5c09d0fc85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b0cf2c787e540b4ba4cd27ec04bbd20'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:16.933708 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dac1ef97-756a-4891-b15d-fa5c09d0fc85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b0cf2c787e540b4ba4cd27ec04bbd20'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:16.941168 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dac1ef97-756a-4891-b15d-fa5c09d0fc85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b0cf2c787e540b4ba4cd27ec04bbd20'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:16.955033 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dac1ef97-756a-4891-b15d-fa5c09d0fc85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b0cf2c787e540b4ba4cd27ec04bbd20'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:16.955804 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dac1ef97-756a-4891-b15d-fa5c09d0fc85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b0cf2c787e540b4ba4cd27ec04bbd20'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:16.962709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1446/2921] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:21:16 2022] DELETE /identity/v3/users/5b0cf2c787e540b4ba4cd27ec04bbd20 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:21:17.002208 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2a0e3-5527-4941-b85f-0c1776e28caf admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:17.003714 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b5b2a0e3-5527-4941-b85f-0c1776e28caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:17.068226 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2a0e3-5527-4941-b85f-0c1776e28caf admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:17.123468 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28351e73-d713-451f-8cd2-4d92e9f78335 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:17.124291 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28351e73-d713-451f-8cd2-4d92e9f78335 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:17.125103 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28351e73-d713-451f-8cd2-4d92e9f78335 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:17.125103 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28351e73-d713-451f-8cd2-4d92e9f78335 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:17.130289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-28351e73-d713-451f-8cd2-4d92e9f78335 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:17.141674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28351e73-d713-451f-8cd2-4d92e9f78335 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:17.143473 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28351e73-d713-451f-8cd2-4d92e9f78335 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:17.155598 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1476/2922] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:21:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:21:17.267076 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1ef97-756a-4891-b15d-fa5c09d0fc85 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:17.268717 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dac1ef97-756a-4891-b15d-fa5c09d0fc85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:17.496251 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1ef97-756a-4891-b15d-fa5c09d0fc85 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:17.502144 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1edcec-bbe5-458e-98c6-3dcfac1d9b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:17.503247 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1edcec-bbe5-458e-98c6-3dcfac1d9b74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:17.503525 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1edcec-bbe5-458e-98c6-3dcfac1d9b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:17.503800 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1edcec-bbe5-458e-98c6-3dcfac1d9b74 admin admin] PATH_INFO: `/v3/projects/c35d3a62e24d40eb923d082a8b4ae874` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:17.505509 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1edcec-bbe5-458e-98c6-3dcfac1d9b74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c35d3a62e24d40eb923d082a8b4ae874)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:17.507951 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1edcec-bbe5-458e-98c6-3dcfac1d9b74 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:17.563793 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1d1edcec-bbe5-458e-98c6-3dcfac1d9b74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c35d3a62e24d40eb923d082a8b4ae874', 'request_id': 'req-1d1edcec-bbe5-458e-98c6-3dcfac1d9b74'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:17.565253 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:21:17.570877 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1d1edcec-bbe5-458e-98c6-3dcfac1d9b74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c35d3a62e24d40eb923d082a8b4ae874', 'request_id': 'req-1d1edcec-bbe5-458e-98c6-3dcfac1d9b74'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:17.579789 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1d1edcec-bbe5-458e-98c6-3dcfac1d9b74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c35d3a62e24d40eb923d082a8b4ae874', 'request_id': 'req-1d1edcec-bbe5-458e-98c6-3dcfac1d9b74'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:17.592659 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1d1edcec-bbe5-458e-98c6-3dcfac1d9b74 admin admin] The token cache is being invalidate because project c35d3a62e24d40eb923d082a8b4ae874 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:21:17.593061 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1d1edcec-bbe5-458e-98c6-3dcfac1d9b74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:17.593552 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1d1edcec-bbe5-458e-98c6-3dcfac1d9b74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:17.604296 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1447/2923] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:21:17 2022] DELETE /identity/v3/projects/c35d3a62e24d40eb923d082a8b4ae874 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:21:17.605289 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5d70b-1cdb-4bc0-8e02-70a7e9005ffe None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:17.605386 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5d70b-1cdb-4bc0-8e02-70a7e9005ffe None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:17.605597 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5d70b-1cdb-4bc0-8e02-70a7e9005ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:17.627379 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-5be5d70b-1cdb-4bc0-8e02-70a7e9005ffe None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:21:17.629497 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-5be5d70b-1cdb-4bc0-8e02-70a7e9005ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:17.642988 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:21:17.714802 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1477/2924] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:21:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 31 15:21:17.720950 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da418d-e105-4a6b-b689-8b654cfcadb4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:17.721242 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da418d-e105-4a6b-b689-8b654cfcadb4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:17.721519 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da418d-e105-4a6b-b689-8b654cfcadb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:17.748344 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c8da418d-e105-4a6b-b689-8b654cfcadb4 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:21:17.749559 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c8da418d-e105-4a6b-b689-8b654cfcadb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:17.759641 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1448/2925] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:21:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:21:17.764280 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5d70b-1cdb-4bc0-8e02-70a7e9005ffe None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:17.776916 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5d70b-1cdb-4bc0-8e02-70a7e9005ffe None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:17.783060 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660f244-bcfb-4af4-a646-145c4fcb69a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:17.784075 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660f244-bcfb-4af4-a646-145c4fcb69a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:17.784644 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660f244-bcfb-4af4-a646-145c4fcb69a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:17.785162 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660f244-bcfb-4af4-a646-145c4fcb69a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:17.786672 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660f244-bcfb-4af4-a646-145c4fcb69a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 31 15:21:17.787258 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660f244-bcfb-4af4-a646-145c4fcb69a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:17.790718 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660f244-bcfb-4af4-a646-145c4fcb69a6 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:17.806810 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1478/2926] 10.222.0.67 () {58 vars in 1152 bytes} [Tue May 31 15:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 31 15:21:17.809944 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da418d-e105-4a6b-b689-8b654cfcadb4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:17.821149 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da418d-e105-4a6b-b689-8b654cfcadb4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:17.826070 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca2231-0816-475c-bd53-2e4839f5640e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:17.826783 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca2231-0816-475c-bd53-2e4839f5640e admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:17.827072 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca2231-0816-475c-bd53-2e4839f5640e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:17.827355 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca2231-0816-475c-bd53-2e4839f5640e admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:17.828053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca2231-0816-475c-bd53-2e4839f5640e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:17.830554 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca2231-0816-475c-bd53-2e4839f5640e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:17.869187 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1449/2927] 10.222.0.67 () {60 vars in 1152 bytes} [Tue May 31 15:21:17 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:21:17.870502 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660f244-bcfb-4af4-a646-145c4fcb69a6 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:17.882342 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660f244-bcfb-4af4-a646-145c4fcb69a6 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:17.951325 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71becd20-2fcf-4956-a150-900f78e5b4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:17.952011 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71becd20-2fcf-4956-a150-900f78e5b4fe admin admin] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:17.952373 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71becd20-2fcf-4956-a150-900f78e5b4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:17.952656 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71becd20-2fcf-4956-a150-900f78e5b4fe admin admin] PATH_INFO: `/v3/users` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:17.954675 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71becd20-2fcf-4956-a150-900f78e5b4fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:17.955541 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71becd20-2fcf-4956-a150-900f78e5b4fe admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:17.981627 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1479/2928] 10.222.0.67 () {60 vars in 1143 bytes} [Tue May 31 15:21:17 2022] POST /identity/v3/users => generated 285 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 31 15:21:17.983852 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca2231-0816-475c-bd53-2e4839f5640e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:17.996744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca2231-0816-475c-bd53-2e4839f5640e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:18.003438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb01a6-e681-4817-a4f5-a4230d08dc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:18.003862 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb01a6-e681-4817-a4f5-a4230d08dc60 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:18.004062 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb01a6-e681-4817-a4f5-a4230d08dc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:18.004345 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb01a6-e681-4817-a4f5-a4230d08dc60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:18.005089 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb01a6-e681-4817-a4f5-a4230d08dc60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:18.007280 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb01a6-e681-4817-a4f5-a4230d08dc60 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:18.020079 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1450/2929] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:21:18.024452 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71becd20-2fcf-4956-a150-900f78e5b4fe admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:18.035383 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71becd20-2fcf-4956-a150-900f78e5b4fe admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:18.040705 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22865361-643a-4820-90af-a6e8455127e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:18.041602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22865361-643a-4820-90af-a6e8455127e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:18.042095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22865361-643a-4820-90af-a6e8455127e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:18.042676 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22865361-643a-4820-90af-a6e8455127e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:18.043467 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22865361-643a-4820-90af-a6e8455127e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:18.045073 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22865361-643a-4820-90af-a6e8455127e3 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:18.056711 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1480/2930] 10.222.0.67 () {58 vars in 1121 bytes} [Tue May 31 15:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 31 15:21:18.061560 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb01a6-e681-4817-a4f5-a4230d08dc60 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:18.072298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb01a6-e681-4817-a4f5-a4230d08dc60 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:18.077722 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23154831-3c16-4335-bc9f-6091ab6f5c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:18.078889 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23154831-3c16-4335-bc9f-6091ab6f5c8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:18.079191 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23154831-3c16-4335-bc9f-6091ab6f5c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:18.079616 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23154831-3c16-4335-bc9f-6091ab6f5c8f admin admin] PATH_INFO: `/v3/projects/88f3bbefc3eb4d15ad4b5a5fcf98cf73/users/cbde102854d54df787c3695f42624b18/roles/9ce8fdecf52d482d894fdeec4f530624` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:18.096474 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23154831-3c16-4335-bc9f-6091ab6f5c8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=88f3bbefc3eb4d15ad4b5a5fcf98cf73, user_id=cbde102854d54df787c3695f42624b18, role_id=9ce8fdecf52d482d894fdeec4f530624)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:18.099130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23154831-3c16-4335-bc9f-6091ab6f5c8f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:18.114839 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1451/2931] 10.222.0.67 () {60 vars in 1482 bytes} [Tue May 31 15:21:18 2022] PUT /identity/v3/projects/88f3bbefc3eb4d15ad4b5a5fcf98cf73/users/cbde102854d54df787c3695f42624b18/roles/9ce8fdecf52d482d894fdeec4f530624 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:21:18.125752 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a559f32-7175-4185-8a45-9edb2ce9ecb3 None None] REQUEST_METHOD: `POST` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:18.126666 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a559f32-7175-4185-8a45-9edb2ce9ecb3 None None] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:18.126666 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a559f32-7175-4185-8a45-9edb2ce9ecb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:18.159588 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.auth.core [None req-0a559f32-7175-4185-8a45-9edb2ce9ecb3 None None] MFA Rules not processed for user `cbde102854d54df787c3695f42624b18`. Rule list: `[]` (Enabled: `True`). {{(pid=80234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:21:18.160926 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-0a559f32-7175-4185-8a45-9edb2ce9ecb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:18.224830 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1481/2932] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:21:18 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:21:18.236822 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23154831-3c16-4335-bc9f-6091ab6f5c8f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:18.238550 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-23154831-3c16-4335-bc9f-6091ab6f5c8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:18.317980 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23154831-3c16-4335-bc9f-6091ab6f5c8f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:18.321827 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02397b7-93ef-4b50-a807-ce1534f23089 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8a4b0dd4e21414d91fe7bed478d15e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:18.322482 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02397b7-93ef-4b50-a807-ce1534f23089 service glance] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:18.322803 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02397b7-93ef-4b50-a807-ce1534f23089 service glance] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:18.323087 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02397b7-93ef-4b50-a807-ce1534f23089 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:18.331490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02397b7-93ef-4b50-a807-ce1534f23089 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:18.332909 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02397b7-93ef-4b50-a807-ce1534f23089 service glance] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:18.342600 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1452/2933] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:21:18 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:21:28.221774 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:21:28.337813 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:DELETE=3,keystone:SELECT=68,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:21:58.685534 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a559f32-7175-4185-8a45-9edb2ce9ecb3 None None] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:58.755213 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a559f32-7175-4185-8a45-9edb2ce9ecb3 None None] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:58.760017 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ee04a-4374-46f7-8476-af4f31ad1db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:58.760973 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ee04a-4374-46f7-8476-af4f31ad1db7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:58.761325 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ee04a-4374-46f7-8476-af4f31ad1db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:58.762095 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ee04a-4374-46f7-8476-af4f31ad1db7 admin admin] PATH_INFO: `/v3/users/cbde102854d54df787c3695f42624b18` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:58.763442 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ee04a-4374-46f7-8476-af4f31ad1db7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbde102854d54df787c3695f42624b18)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:58.765387 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ee04a-4374-46f7-8476-af4f31ad1db7 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:58.820127 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9f3ee04a-4374-46f7-8476-af4f31ad1db7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbde102854d54df787c3695f42624b18'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:58.830221 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9f3ee04a-4374-46f7-8476-af4f31ad1db7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbde102854d54df787c3695f42624b18'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:58.830860 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9f3ee04a-4374-46f7-8476-af4f31ad1db7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbde102854d54df787c3695f42624b18'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:58.846502 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9f3ee04a-4374-46f7-8476-af4f31ad1db7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbde102854d54df787c3695f42624b18'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:58.847291 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-9f3ee04a-4374-46f7-8476-af4f31ad1db7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbde102854d54df787c3695f42624b18'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:58.860550 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1482/2934] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:21:58 2022] DELETE /identity/v3/users/cbde102854d54df787c3695f42624b18 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:21:58.865070 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb255a4-a007-454d-b97c-8f31ca872322 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:58.865598 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb255a4-a007-454d-b97c-8f31ca872322 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:58.865696 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb255a4-a007-454d-b97c-8f31ca872322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:58.892175 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0bb255a4-a007-454d-b97c-8f31ca872322 None None] MFA Rules not processed for user `c01eab2db7454d578ce76aa7d022d460`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:21:58.893490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0bb255a4-a007-454d-b97c-8f31ca872322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:58.959234 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1453/2935] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:21:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:21:58.969593 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ee04a-4374-46f7-8476-af4f31ad1db7 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:58.971138 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-9f3ee04a-4374-46f7-8476-af4f31ad1db7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:59.036308 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ee04a-4374-46f7-8476-af4f31ad1db7 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:59.040170 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe98466-c8c2-4071-b710-ee71a533832f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:59.040846 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe98466-c8c2-4071-b710-ee71a533832f service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:59.041129 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe98466-c8c2-4071-b710-ee71a533832f service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:59.041407 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe98466-c8c2-4071-b710-ee71a533832f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:59.053227 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe98466-c8c2-4071-b710-ee71a533832f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:59.054762 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe98466-c8c2-4071-b710-ee71a533832f service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:59.065632 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1483/2936] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:21:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:21:59.384490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb255a4-a007-454d-b97c-8f31ca872322 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:21:59.388294 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0bb255a4-a007-454d-b97c-8f31ca872322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:21:59.413976 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb255a4-a007-454d-b97c-8f31ca872322 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:21:59.418941 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b957e-5505-4360-bf08-c40e3acbc993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:21:59.419838 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b957e-5505-4360-bf08-c40e3acbc993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:21:59.421013 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b957e-5505-4360-bf08-c40e3acbc993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:21:59.421694 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b957e-5505-4360-bf08-c40e3acbc993 admin admin] PATH_INFO: `/v3/projects/88f3bbefc3eb4d15ad4b5a5fcf98cf73` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:21:59.423006 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b957e-5505-4360-bf08-c40e3acbc993 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88f3bbefc3eb4d15ad4b5a5fcf98cf73)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:21:59.424956 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b957e-5505-4360-bf08-c40e3acbc993 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:21:59.488840 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-583b957e-5505-4360-bf08-c40e3acbc993 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88f3bbefc3eb4d15ad4b5a5fcf98cf73', 'request_id': 'req-583b957e-5505-4360-bf08-c40e3acbc993'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:59.497744 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-583b957e-5505-4360-bf08-c40e3acbc993 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88f3bbefc3eb4d15ad4b5a5fcf98cf73', 'request_id': 'req-583b957e-5505-4360-bf08-c40e3acbc993'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:59.505130 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-583b957e-5505-4360-bf08-c40e3acbc993 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88f3bbefc3eb4d15ad4b5a5fcf98cf73', 'request_id': 'req-583b957e-5505-4360-bf08-c40e3acbc993'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:59.523969 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-583b957e-5505-4360-bf08-c40e3acbc993 admin admin] The token cache is being invalidate because project 88f3bbefc3eb4d15ad4b5a5fcf98cf73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:21:59.524298 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-583b957e-5505-4360-bf08-c40e3acbc993 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:59.524940 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-583b957e-5505-4360-bf08-c40e3acbc993 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:21:59.527053 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1454/2937] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:21:59 2022] DELETE /identity/v3/projects/88f3bbefc3eb4d15ad4b5a5fcf98cf73 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:22:09.060772 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:22:09.520204 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:22:21.912508 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe98466-c8c2-4071-b710-ee71a533832f service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:22:21.914292 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-afe98466-c8c2-4071-b710-ee71a533832f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:22:22.033965 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe98466-c8c2-4071-b710-ee71a533832f service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:22:22.092667 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57061d-0f9b-4703-b9fa-b79e6803f2a7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c1c13515a8b41a781478dc65e6a88b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:22:22.093342 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57061d-0f9b-4703-b9fa-b79e6803f2a7 service placement] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:22:22.093625 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57061d-0f9b-4703-b9fa-b79e6803f2a7 service placement] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:22:22.093950 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57061d-0f9b-4703-b9fa-b79e6803f2a7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:22:22.095743 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6d57061d-0f9b-4703-b9fa-b79e6803f2a7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:22:22.108164 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57061d-0f9b-4703-b9fa-b79e6803f2a7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:22:22.109742 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57061d-0f9b-4703-b9fa-b79e6803f2a7 service placement] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:22:22.120177 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1484/2938] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:22:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:22:32.115334 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=71 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:22:43.186127 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae7e40-782a-4d24-bc21-59fa1a72bd05 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:22:43.186637 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae7e40-782a-4d24-bc21-59fa1a72bd05 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:22:43.186759 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae7e40-782a-4d24-bc21-59fa1a72bd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:22:43.204639 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6cae7e40-782a-4d24-bc21-59fa1a72bd05 None None] MFA Rules not processed for user `65894910ee8647f7b32395c0017d3562`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 31 15:22:43.205833 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6cae7e40-782a-4d24-bc21-59fa1a72bd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:22:43.260757 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1455/2939] 10.222.0.67 () {58 vars in 957 bytes} [Tue May 31 15:22:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 31 15:22:53.253883 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=25 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:23:19.758779 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57061d-0f9b-4703-b9fa-b79e6803f2a7 service placement] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:23:19.760253 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6d57061d-0f9b-4703-b9fa-b79e6803f2a7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:23:19.815958 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57061d-0f9b-4703-b9fa-b79e6803f2a7 service placement] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:23:19.819677 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd1c90-6170-4039-b95b-db93a96fcb43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:23:19.820329 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd1c90-6170-4039-b95b-db93a96fcb43 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:23:19.820612 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd1c90-6170-4039-b95b-db93a96fcb43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:23:19.820891 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd1c90-6170-4039-b95b-db93a96fcb43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:23:19.822743 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-59bd1c90-6170-4039-b95b-db93a96fcb43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:23:19.943213 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd1c90-6170-4039-b95b-db93a96fcb43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:23:19.944715 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd1c90-6170-4039-b95b-db93a96fcb43 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:23:19.957463 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1485/2940] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:23:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:23:21.140219 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae7e40-782a-4d24-bc21-59fa1a72bd05 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:23:21.150157 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae7e40-782a-4d24-bc21-59fa1a72bd05 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:23:21.155621 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc408d-b886-44a7-b65e-f84458e609d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:23:21.156335 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc408d-b886-44a7-b65e-f84458e609d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:23:21.156623 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc408d-b886-44a7-b65e-f84458e609d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:23:21.156905 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc408d-b886-44a7-b65e-f84458e609d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:23:21.158796 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-63dc408d-b886-44a7-b65e-f84458e609d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:23:21.173303 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc408d-b886-44a7-b65e-f84458e609d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:23:21.174633 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc408d-b886-44a7-b65e-f84458e609d7 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:23:21.186931 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1456/2941] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:23:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:23:29.951573 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=68 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:23:31.181734 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=6 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:23:49.144032 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd1c90-6170-4039-b95b-db93a96fcb43 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:23:49.145505 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-59bd1c90-6170-4039-b95b-db93a96fcb43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:23:49.213332 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd1c90-6170-4039-b95b-db93a96fcb43 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:23:49.217821 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c99b26-876e-40c5-9ff3-35c49a21171c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:23:49.218763 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c99b26-876e-40c5-9ff3-35c49a21171c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:23:49.219077 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c99b26-876e-40c5-9ff3-35c49a21171c admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:23:49.219319 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c99b26-876e-40c5-9ff3-35c49a21171c admin admin] PATH_INFO: `/v3/users/65894910ee8647f7b32395c0017d3562` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:23:49.220582 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c99b26-876e-40c5-9ff3-35c49a21171c admin admin] RBAC: Authorizing `identity:delete_user(user_id=65894910ee8647f7b32395c0017d3562)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:23:49.223258 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c99b26-876e-40c5-9ff3-35c49a21171c admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:23:49.275503 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a2c99b26-876e-40c5-9ff3-35c49a21171c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65894910ee8647f7b32395c0017d3562'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:23:49.283023 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a2c99b26-876e-40c5-9ff3-35c49a21171c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65894910ee8647f7b32395c0017d3562'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:23:49.284181 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a2c99b26-876e-40c5-9ff3-35c49a21171c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65894910ee8647f7b32395c0017d3562'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:23:49.295850 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a2c99b26-876e-40c5-9ff3-35c49a21171c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65894910ee8647f7b32395c0017d3562'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:23:49.296481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-a2c99b26-876e-40c5-9ff3-35c49a21171c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65894910ee8647f7b32395c0017d3562'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:23:49.308960 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1486/2942] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:23:49 2022] DELETE /identity/v3/users/65894910ee8647f7b32395c0017d3562 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:23:49.484391 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc408d-b886-44a7-b65e-f84458e609d7 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:23:49.498892 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc408d-b886-44a7-b65e-f84458e609d7 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:23:49.565563 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d9f95-36f5-4625-b3ad-2edd6ac1c697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:23:49.566448 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d9f95-36f5-4625-b3ad-2edd6ac1c697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:23:49.566773 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d9f95-36f5-4625-b3ad-2edd6ac1c697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:23:49.567050 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d9f95-36f5-4625-b3ad-2edd6ac1c697 admin admin] PATH_INFO: `/v3/projects/2dac67cb14e24c41b4add3b40a103fea` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:23:49.568278 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d9f95-36f5-4625-b3ad-2edd6ac1c697 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dac67cb14e24c41b4add3b40a103fea)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:23:49.570244 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d9f95-36f5-4625-b3ad-2edd6ac1c697 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:23:49.621945 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-055d9f95-36f5-4625-b3ad-2edd6ac1c697 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dac67cb14e24c41b4add3b40a103fea', 'request_id': 'req-055d9f95-36f5-4625-b3ad-2edd6ac1c697'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:23:49.627964 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-055d9f95-36f5-4625-b3ad-2edd6ac1c697 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dac67cb14e24c41b4add3b40a103fea', 'request_id': 'req-055d9f95-36f5-4625-b3ad-2edd6ac1c697'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:23:49.634375 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-055d9f95-36f5-4625-b3ad-2edd6ac1c697 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dac67cb14e24c41b4add3b40a103fea', 'request_id': 'req-055d9f95-36f5-4625-b3ad-2edd6ac1c697'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:23:49.649060 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-055d9f95-36f5-4625-b3ad-2edd6ac1c697 admin admin] The token cache is being invalidate because project 2dac67cb14e24c41b4add3b40a103fea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:23:49.649373 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-055d9f95-36f5-4625-b3ad-2edd6ac1c697 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:23:49.650010 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-055d9f95-36f5-4625-b3ad-2edd6ac1c697 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:23:49.651606 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1457/2943] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:23:49 2022] DELETE /identity/v3/projects/2dac67cb14e24c41b4add3b40a103fea => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:23:59.305405 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:23:59.646031 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:24:22.528413 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c99b26-876e-40c5-9ff3-35c49a21171c admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:24:22.529975 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-a2c99b26-876e-40c5-9ff3-35c49a21171c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:24:22.627044 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c99b26-876e-40c5-9ff3-35c49a21171c admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:24:22.695081 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea026831-47f3-4f74-b21a-950a8f56cd70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:24:22.695772 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea026831-47f3-4f74-b21a-950a8f56cd70 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:24:22.696054 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea026831-47f3-4f74-b21a-950a8f56cd70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:24:22.696325 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea026831-47f3-4f74-b21a-950a8f56cd70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:24:22.698317 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ea026831-47f3-4f74-b21a-950a8f56cd70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:24:22.709565 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea026831-47f3-4f74-b21a-950a8f56cd70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:24:22.710972 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea026831-47f3-4f74-b21a-950a8f56cd70 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:24:22.719858 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1487/2944] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:24:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:24:22.821388 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d9f95-36f5-4625-b3ad-2edd6ac1c697 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:24:22.822951 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-055d9f95-36f5-4625-b3ad-2edd6ac1c697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:24:22.842366 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d9f95-36f5-4625-b3ad-2edd6ac1c697 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:24:22.847559 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655acfc0-48c4-416d-bd28-edc99c8f3d47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:24:22.848230 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655acfc0-48c4-416d-bd28-edc99c8f3d47 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:24:22.848520 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655acfc0-48c4-416d-bd28-edc99c8f3d47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:24:22.848804 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655acfc0-48c4-416d-bd28-edc99c8f3d47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:24:22.850729 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-655acfc0-48c4-416d-bd28-edc99c8f3d47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:24:22.863967 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655acfc0-48c4-416d-bd28-edc99c8f3d47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:24:22.866331 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655acfc0-48c4-416d-bd28-edc99c8f3d47 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:24:22.877940 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1458/2945] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:24:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:24:22.972621 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea026831-47f3-4f74-b21a-950a8f56cd70 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:24:22.981654 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea026831-47f3-4f74-b21a-950a8f56cd70 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:24:22.985664 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c202831-10c0-4791-809c-36575bd411a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:24:22.986320 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c202831-10c0-4791-809c-36575bd411a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:24:22.986607 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c202831-10c0-4791-809c-36575bd411a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:24:22.986920 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c202831-10c0-4791-809c-36575bd411a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:24:22.989944 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4c202831-10c0-4791-809c-36575bd411a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:24:23.067586 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c202831-10c0-4791-809c-36575bd411a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:24:23.069045 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c202831-10c0-4791-809c-36575bd411a1 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:24:23.080481 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1488/2946] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:24:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:24:23.145009 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655acfc0-48c4-416d-bd28-edc99c8f3d47 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:24:23.146403 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-655acfc0-48c4-416d-bd28-edc99c8f3d47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:24:23.219121 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655acfc0-48c4-416d-bd28-edc99c8f3d47 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:24:23.225215 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0521c848-7b56-408f-8853-3f6469255422 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c1c13515a8b41a781478dc65e6a88b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:24:23.225974 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521c848-7b56-408f-8853-3f6469255422 service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:24:23.226162 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521c848-7b56-408f-8853-3f6469255422 service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:24:23.226438 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521c848-7b56-408f-8853-3f6469255422 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:24:23.228314 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0521c848-7b56-408f-8853-3f6469255422 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:24:23.241324 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0521c848-7b56-408f-8853-3f6469255422 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:24:23.242959 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0521c848-7b56-408f-8853-3f6469255422 service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:24:23.253181 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1459/2947] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:24:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:24:25.601797 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c202831-10c0-4791-809c-36575bd411a1 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:24:25.604003 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-4c202831-10c0-4791-809c-36575bd411a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:24:25.618555 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c202831-10c0-4791-809c-36575bd411a1 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:24:25.624329 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc3f3a-7938-44aa-bc72-c7cdd3a72d8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:24:25.625010 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc3f3a-7938-44aa-bc72-c7cdd3a72d8b service nova] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:24:25.625296 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc3f3a-7938-44aa-bc72-c7cdd3a72d8b service nova] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:24:25.625569 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc3f3a-7938-44aa-bc72-c7cdd3a72d8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:24:25.627452 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-facc3f3a-7938-44aa-bc72-c7cdd3a72d8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:24:25.640136 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc3f3a-7938-44aa-bc72-c7cdd3a72d8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:24:25.641745 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc3f3a-7938-44aa-bc72-c7cdd3a72d8b service nova] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:24:25.653985 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1489/2948] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:24:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:24:33.248811 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=40 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:24:35.648777 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=95 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:25:23.038184 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0521c848-7b56-408f-8853-3f6469255422 service placement] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:25:23.039725 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0521c848-7b56-408f-8853-3f6469255422 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:25:23.061303 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0521c848-7b56-408f-8853-3f6469255422 service placement] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:25:23.064862 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1f0d1-c268-430c-ae36-436039130846 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:25:23.065511 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1f0d1-c268-430c-ae36-436039130846 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:25:23.065792 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1f0d1-c268-430c-ae36-436039130846 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:25:23.066067 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1f0d1-c268-430c-ae36-436039130846 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:25:23.067879 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-08e1f0d1-c268-430c-ae36-436039130846 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:25:23.077490 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1f0d1-c268-430c-ae36-436039130846 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:25:23.078928 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1f0d1-c268-430c-ae36-436039130846 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:25:23.087507 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1460/2949] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:25:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:25:33.084248 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=10 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:26:10.920988 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc3f3a-7938-44aa-bc72-c7cdd3a72d8b service nova] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:26:10.930936 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc3f3a-7938-44aa-bc72-c7cdd3a72d8b service nova] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:26:10.934437 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb4fe0-fde9-4133-860d-f7d7c1c91973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:26:10.935113 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb4fe0-fde9-4133-860d-f7d7c1c91973 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:26:10.935397 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb4fe0-fde9-4133-860d-f7d7c1c91973 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:26:10.935674 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb4fe0-fde9-4133-860d-f7d7c1c91973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:26:10.937486 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-e0cb4fe0-fde9-4133-860d-f7d7c1c91973 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:26:10.995440 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cb4fe0-fde9-4133-860d-f7d7c1c91973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:26:10.996886 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cb4fe0-fde9-4133-860d-f7d7c1c91973 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:26:11.005772 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1490/2950] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:26:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:26:11.835412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1f0d1-c268-430c-ae36-436039130846 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:26:11.837827 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-08e1f0d1-c268-430c-ae36-436039130846 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:26:11.856206 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1f0d1-c268-430c-ae36-436039130846 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:26:11.862577 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba613da-4ee1-458a-92bf-1683850d3da5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68c6363b60954c3ebf50d2019f5f2fd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:26:11.863276 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba613da-4ee1-458a-92bf-1683850d3da5 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:26:11.864162 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba613da-4ee1-458a-92bf-1683850d3da5 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:26:11.864162 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba613da-4ee1-458a-92bf-1683850d3da5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:26:11.865709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fba613da-4ee1-458a-92bf-1683850d3da5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:26:11.879862 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba613da-4ee1-458a-92bf-1683850d3da5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:26:11.881553 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba613da-4ee1-458a-92bf-1683850d3da5 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:26:11.905510 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1461/2951] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:26:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:26:15.687123 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb4fe0-fde9-4133-860d-f7d7c1c91973 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:26:15.696211 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb4fe0-fde9-4133-860d-f7d7c1c91973 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:26:15.699789 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac170d95-223d-4bfb-b5f5-08c4e3c9abf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8966bc2ea7ff497c84214c9c9e6d53a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:26:15.700434 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac170d95-223d-4bfb-b5f5-08c4e3c9abf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:26:15.700719 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac170d95-223d-4bfb-b5f5-08c4e3c9abf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:26:15.700999 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac170d95-223d-4bfb-b5f5-08c4e3c9abf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:26:15.702856 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ac170d95-223d-4bfb-b5f5-08c4e3c9abf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:26:15.767730 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac170d95-223d-4bfb-b5f5-08c4e3c9abf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:26:15.769172 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac170d95-223d-4bfb-b5f5-08c4e3c9abf9 service neutron] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:26:15.779083 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1491/2952] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:26:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:26:18.864082 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba613da-4ee1-458a-92bf-1683850d3da5 service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:26:18.882480 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba613da-4ee1-458a-92bf-1683850d3da5 service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:26:18.891940 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbda345-8f10-4469-be59-0c6eafcc1563 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c1c13515a8b41a781478dc65e6a88b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:26:18.893149 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbda345-8f10-4469-be59-0c6eafcc1563 service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:26:18.893620 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbda345-8f10-4469-be59-0c6eafcc1563 service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:26:18.894068 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbda345-8f10-4469-be59-0c6eafcc1563 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:26:18.896160 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8fbda345-8f10-4469-be59-0c6eafcc1563 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:26:18.912382 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbda345-8f10-4469-be59-0c6eafcc1563 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:26:18.914857 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbda345-8f10-4469-be59-0c6eafcc1563 service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:26:18.927085 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1462/2953] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:26:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:26:22.092898 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac170d95-223d-4bfb-b5f5-08c4e3c9abf9 service neutron] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:26:22.094529 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-ac170d95-223d-4bfb-b5f5-08c4e3c9abf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:26:22.107919 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac170d95-223d-4bfb-b5f5-08c4e3c9abf9 service neutron] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:26:22.112688 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206beff-7ad3-4a57-b2db-3ca6bf739c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:26:22.113571 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206beff-7ad3-4a57-b2db-3ca6bf739c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:26:22.113854 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206beff-7ad3-4a57-b2db-3ca6bf739c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:26:22.114127 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206beff-7ad3-4a57-b2db-3ca6bf739c51 admin admin] PATH_INFO: `/v3/users/6a8c202f3b4a424cb6565fc8a6abd15b` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:26:22.115350 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206beff-7ad3-4a57-b2db-3ca6bf739c51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a8c202f3b4a424cb6565fc8a6abd15b)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:26:22.117752 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206beff-7ad3-4a57-b2db-3ca6bf739c51 admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:26:22.174315 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6206beff-7ad3-4a57-b2db-3ca6bf739c51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a8c202f3b4a424cb6565fc8a6abd15b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:26:22.183084 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6206beff-7ad3-4a57-b2db-3ca6bf739c51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a8c202f3b4a424cb6565fc8a6abd15b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:26:22.183761 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6206beff-7ad3-4a57-b2db-3ca6bf739c51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a8c202f3b4a424cb6565fc8a6abd15b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:26:22.197219 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6206beff-7ad3-4a57-b2db-3ca6bf739c51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a8c202f3b4a424cb6565fc8a6abd15b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:26:22.198171 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-6206beff-7ad3-4a57-b2db-3ca6bf739c51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a8c202f3b4a424cb6565fc8a6abd15b'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:26:22.212000 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1492/2954] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:26:22 2022] DELETE /identity/v3/users/6a8c202f3b4a424cb6565fc8a6abd15b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:26:22.380654 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbda345-8f10-4469-be59-0c6eafcc1563 service placement] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:26:22.397170 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbda345-8f10-4469-be59-0c6eafcc1563 service placement] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:26:22.453637 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2dc19-c44e-45fe-93dd-3e3fe0ba9e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:26:22.454401 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2dc19-c44e-45fe-93dd-3e3fe0ba9e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:26:22.454709 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2dc19-c44e-45fe-93dd-3e3fe0ba9e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:26:22.454986 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2dc19-c44e-45fe-93dd-3e3fe0ba9e36 admin admin] PATH_INFO: `/v3/projects/e24193a7426547f5895f8b8074e46875` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:26:22.456139 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2dc19-c44e-45fe-93dd-3e3fe0ba9e36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e24193a7426547f5895f8b8074e46875)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:26:22.458116 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2dc19-c44e-45fe-93dd-3e3fe0ba9e36 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:26:22.503954 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-23a2dc19-c44e-45fe-93dd-3e3fe0ba9e36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e24193a7426547f5895f8b8074e46875', 'request_id': 'req-23a2dc19-c44e-45fe-93dd-3e3fe0ba9e36'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:26:22.509366 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-23a2dc19-c44e-45fe-93dd-3e3fe0ba9e36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e24193a7426547f5895f8b8074e46875', 'request_id': 'req-23a2dc19-c44e-45fe-93dd-3e3fe0ba9e36'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:26:22.515332 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-23a2dc19-c44e-45fe-93dd-3e3fe0ba9e36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e24193a7426547f5895f8b8074e46875', 'request_id': 'req-23a2dc19-c44e-45fe-93dd-3e3fe0ba9e36'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:26:22.528182 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-23a2dc19-c44e-45fe-93dd-3e3fe0ba9e36 admin admin] The token cache is being invalidate because project e24193a7426547f5895f8b8074e46875 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:26:22.528465 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-23a2dc19-c44e-45fe-93dd-3e3fe0ba9e36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:26:22.529342 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-23a2dc19-c44e-45fe-93dd-3e3fe0ba9e36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:26:22.530916 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1463/2955] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:26:22 2022] DELETE /identity/v3/projects/e24193a7426547f5895f8b8074e46875 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:26:22.534938 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206beff-7ad3-4a57-b2db-3ca6bf739c51 admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:26:22.536329 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-6206beff-7ad3-4a57-b2db-3ca6bf739c51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:26:22.546005 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:26:22.550288 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206beff-7ad3-4a57-b2db-3ca6bf739c51 admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:26:22.607750 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c1edf-4958-43de-ab35-d7623c02c5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:26:22.608629 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c1edf-4958-43de-ab35-d7623c02c5dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:26:22.608912 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c1edf-4958-43de-ab35-d7623c02c5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:26:22.609191 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c1edf-4958-43de-ab35-d7623c02c5dc admin admin] PATH_INFO: `/v3/users/62c33a95fb124fb78697cfed4f1fe976` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:26:22.610381 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c1edf-4958-43de-ab35-d7623c02c5dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=62c33a95fb124fb78697cfed4f1fe976)` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:26:22.612343 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c1edf-4958-43de-ab35-d7623c02c5dc admin admin] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:26:22.658107 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c85c1edf-4958-43de-ab35-d7623c02c5dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62c33a95fb124fb78697cfed4f1fe976'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:26:22.665601 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c85c1edf-4958-43de-ab35-d7623c02c5dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62c33a95fb124fb78697cfed4f1fe976'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:26:22.666212 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c85c1edf-4958-43de-ab35-d7623c02c5dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62c33a95fb124fb78697cfed4f1fe976'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:26:22.677997 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c85c1edf-4958-43de-ab35-d7623c02c5dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62c33a95fb124fb78697cfed4f1fe976'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:26:22.678669 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.notifications [None req-c85c1edf-4958-43de-ab35-d7623c02c5dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62c33a95fb124fb78697cfed4f1fe976'} {{(pid=80234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:26:22.690593 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1493/2956] 10.222.0.67 () {58 vars in 1223 bytes} [Tue May 31 15:26:22 2022] DELETE /identity/v3/users/62c33a95fb124fb78697cfed4f1fe976 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:26:23.139011 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2dc19-c44e-45fe-93dd-3e3fe0ba9e36 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:26:23.141246 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-23a2dc19-c44e-45fe-93dd-3e3fe0ba9e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:26:23.156412 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2dc19-c44e-45fe-93dd-3e3fe0ba9e36 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:26:23.215402 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d3d78-c6c8-4776-b20f-a13dc8967a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c01eab2db7454d578ce76aa7d022d460', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '929ceca16868457ca23cb37daf0ee36d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:26:23.216270 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d3d78-c6c8-4776-b20f-a13dc8967a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:26:23.216551 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d3d78-c6c8-4776-b20f-a13dc8967a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:26:23.216828 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d3d78-c6c8-4776-b20f-a13dc8967a81 admin admin] PATH_INFO: `/v3/projects/12d69bfb660f48efb90d7fff9bea45d1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:26:23.218000 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d3d78-c6c8-4776-b20f-a13dc8967a81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12d69bfb660f48efb90d7fff9bea45d1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:26:23.219930 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d3d78-c6c8-4776-b20f-a13dc8967a81 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:26:23.226377 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:26:23.268034 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-471d3d78-c6c8-4776-b20f-a13dc8967a81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12d69bfb660f48efb90d7fff9bea45d1', 'request_id': 'req-471d3d78-c6c8-4776-b20f-a13dc8967a81'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:26:23.274268 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-471d3d78-c6c8-4776-b20f-a13dc8967a81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12d69bfb660f48efb90d7fff9bea45d1', 'request_id': 'req-471d3d78-c6c8-4776-b20f-a13dc8967a81'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:26:23.280902 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-471d3d78-c6c8-4776-b20f-a13dc8967a81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12d69bfb660f48efb90d7fff9bea45d1', 'request_id': 'req-471d3d78-c6c8-4776-b20f-a13dc8967a81'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:26:23.293334 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-471d3d78-c6c8-4776-b20f-a13dc8967a81 admin admin] The token cache is being invalidate because project 12d69bfb660f48efb90d7fff9bea45d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 31 15:26:23.293621 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-471d3d78-c6c8-4776-b20f-a13dc8967a81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:26:23.294260 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-471d3d78-c6c8-4776-b20f-a13dc8967a81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 31 15:26:23.295825 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1464/2957] 10.222.0.67 () {58 vars in 1232 bytes} [Tue May 31 15:26:23 2022] DELETE /identity/v3/projects/12d69bfb660f48efb90d7fff9bea45d1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 31 15:26:32.687087 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=51,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:26:33.290369 dv-cinder-839793-14-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 31 15:28:21.278590 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c1edf-4958-43de-ab35-d7623c02c5dc admin admin] Authenticating user token {{(pid=80234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 31 15:28:21.280300 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-c85c1edf-4958-43de-ab35-d7623c02c5dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:28:21.348124 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c1edf-4958-43de-ab35-d7623c02c5dc admin admin] Validating token access rules against request {{(pid=80234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 31 15:28:21.410240 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e944cf-22fc-4a3e-ac9a-0b4f74086ccc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c1c13515a8b41a781478dc65e6a88b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f965a93a2f674598890bd8cafededb2c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 31 15:28:21.410957 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e944cf-22fc-4a3e-ac9a-0b4f74086ccc service placement] REQUEST_METHOD: `GET` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 31 15:28:21.411250 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e944cf-22fc-4a3e-ac9a-0b4f74086ccc service placement] SCRIPT_NAME: `/identity` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 31 15:28:21.411532 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e944cf-22fc-4a3e-ac9a-0b4f74086ccc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 31 15:28:21.413369 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.fernet_utils [None req-b8e944cf-22fc-4a3e-ac9a-0b4f74086ccc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 31 15:28:21.425814 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e944cf-22fc-4a3e-ac9a-0b4f74086ccc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 31 15:28:21.427456 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e944cf-22fc-4a3e-ac9a-0b4f74086ccc service placement] RBAC: Authorization granted {{(pid=80234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 31 15:28:21.443473 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: [pid: 80234|app: 0|req: 1494/2958] 10.222.0.67 () {60 vars in 1365 bytes} [Tue May 31 15:28:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 31 15:28:31.438602 dv-cinder-839793-14-iscsi devstack@keystone.service[80234]: DEBUG dbcounter [-] [80234] Writing DB stats keystone:SELECT=58 {{(pid=80234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}